Some Questions On Bridge Mode For MCTPD

[santoshpuranik]

Hi,

This is not really an "issue" with this repo, rather a starting point for discussions around a requirement I am currently analyzing.

I have a MCTP bridge device (a BMC running Linux). The bridge directly manages two other MCTP devices connected over a USB bus downstream. Upstream, the bridge is connected (as a USB device) to a USB host (another BMC) that needs to communicate to these two downstream devices. Here's a diagram that depicts this:

graph TD
    USB_Host["MC1 (USB Host)"]
    Bridge["MC2 - MCTP Bridge (Linux)"]
    Dev1["MCTP Device 1"]
    Dev2["MCTP Device 2"]

    USB_Host -->|"USB Device"| Bridge
    Bridge -->|"USB Host"| Dev1
    Bridge -->|"USB Host"| Dev2

Loading

An additional requirement is that MC1 should only be allowed to Tx/Rx certain MCTP message types, effectively, MC2 acts as a firewall for the MCTP traffic flowing through it. I have the following questions:

• Are there plans to upstream the USB MCTP gadget driver? IIRC that was used to test the host side driver before upstreaming?
• The MCTP spec itself does not talk about firewalling traffic, but given that we have the AF_MCTP socket for the netdev, would it be reasonable to implement a socket filter on it externally to achieve this firewall?
• For the bridge specific MCTP control commands, implement a bridge mode in the mctpd daemon (on the mctpgx net i/f) on MC2 that can respond to the mandatory set of control commands from MC1 -- most significantly providing it a routing table that includes entries for the two downstream devices.

Any thoughts on if this all makes sense and other possible approaches to tackle these requirements?

Regards,
Santosh

[jk-ozlabs]

Are there plans to upstream the USB MCTP gadget driver? IIRC that was used to test the host side driver before upstreaming?

Yes - it is on my todo list, but there are quite a few things in front of it :)

We now have more configurable ways to do USB testing (mctp-dev, mainly), but it would be handy to have the gadget driver to more "production" use-cases.

The MCTP spec itself does not talk about firewalling traffic, but given that we have the AF_MCTP socket for the netdev, would it be reasonable to implement a socket filter on it externally to achieve this firewall?

I'm not sure how intentional the reference to a "socket filter" here is - there is no socket involved for forwarded packets. However, it would certainly be feasible to implement some checking of forwarded packets:

1. In-kernel: where you would want a netfilter-like approach (and corresponding toolset). For your use-case, you really want message-based (rather than packet-based) filtering, as you're checking the message-type field, so would either need to perform some sort of reassembly (not currently done for forwarded packets, as per the spec), or implement stateful packet filtering
2. As a userspace proxy: where you would have your endpoints on separate MCTP networks (so you have isolation by default), and a userspace process receiving (reassembled) MCTP messages, performing the required checks, and NAT-ing to their correct destinations.

(1) is certainly a more solid long-term approach, but requires a lot of design work.

If the addressing semantics of (2) don't suit, you could always implement this as a tun-based bridge between the two networks, and have it completely transparent, but the state-tracking gets a little more complex there.

For the bridge specific MCTP control commands, implement a bridge mode in the mctpd daemon (on the mctpgx net i/f) on MC2 that can respond to the mandatory set of control commands from MC1 -- most significantly providing it a routing table that includes entries for the two downstream devices.

Yep, would be great to have the bridge-mandatory commands implemented, but I'm not clear on why you need Get Routing Table Entries here - previous discussions where folks have suspected they needed that information have turned out that it is generally unnecessary.

If you do need this, then mctpd would then need to be aware of your potentially-cross-network forwarding setup.

[santoshpuranik]

Thank you for the response!

Are there plans to upstream the USB MCTP gadget driver? IIRC that was used to test the host side driver before upstreaming?

Yes - it is on my todo list, but there are quite a few things in front of it :)

Ack. :)

The MCTP spec itself does not talk about firewalling traffic, but given that we have the AF_MCTP socket for the netdev, would it be reasonable to implement a socket filter on it externally to achieve this firewall?

I'm not sure how intentional the reference to a "socket filter" here is - there is no socket involved for forwarded packets. However, it would certainly be feasible to implement some checking of forwarded packets:

Ah, indeed. There would be no userspace involved in bridging the packets - so no socket filter needed (I was referring to implementing a SO_ATTACH_FILTER on the socket opened by the userspace, but as you point out, that is just not involved when the kernel is routing packets internally)

1. In-kernel: where you would want a netfilter-like approach (and corresponding toolset). For your use-case, you really want message-based (rather than packet-based) filtering, as you're checking the message-type field, so would either need to perform some sort of reassembly (not currently done for forwarded packets, as per the spec), or implement stateful packet filtering

2. As a userspace proxy: where you would have your endpoints on separate MCTP networks (so you have isolation by default), and a userspace process receiving (reassembled) MCTP messages, performing the required checks, and NAT-ing to their correct destinations.

(1) is certainly a more solid long-term approach, but requires a lot of design work.

If the addressing semantics of (2) don't suit, you could always implement this as a tun-based bridge between the two networks, and have it completely transparent, but the state-tracking gets a little more complex there.

I was thinking of (1) as well -- could you elaborate a little on the userspace toolset that would be needed here to configure the filter. I am thinking the netfilter would have to be attached on the mctpusbgX netdev, right (as opposed to the other mctpusbX interfaces - as they need to accept all message types). Also ack on the stateful-ness of the filter - it would have to maintain the state with, perhaps {srd eid, dst eid, tag} tuple that was in the first packet of - possibly - a multi-packet message?

For the bridge specific MCTP control commands, implement a bridge mode in the mctpd daemon (on the mctpgx net i/f) on MC2 that can respond to the mandatory set of control commands from MC1 -- most significantly providing it a routing table that includes entries for the two downstream devices.

Yep, would be great to have the bridge-mandatory commands implemented, but I'm not clear on why you need Get Routing Table Entries here - previous discussions where folks have suspected they needed that information have turned out that it is generally unnecessary.

If you do need this, then mctpd would then need to be aware of your potentially-cross-network forwarding setup.

GetRoutingTable would be the way for MC1 to discover the endpoints downstream to MC2, right? The EID assignments for those devices are to happen via MC2 (So MC2 is not going to request an EID pool from MC1). The spec also lists GetRoutingTable as a mandatory command for bridges to accept.

[jk-ozlabs]

I was thinking of (1) as well -- could you elaborate a little on the userspace toolset that would be needed here to configure the filter.

Ideally, this would just be adding MCTP support to the existing nft utility. This assumes that the kernel side is implemented similarly to the existing netfilter hooks, but there may be a lot of ip4/ip6 assumptions to unthread as part of doing so.

I am thinking the netfilter would have to be attached on the mctpusbgX netdev

The netfilter hooks do have access to the device in specific tables. This case would be a FORWARD-like table, so you have both the ingress and egress interfaces available for rule matches.

GetRoutingTable would be the way for MC1 to discover the endpoints downstream to MC2, right?

No, we have a lot of discussion on that in #71. In summary: Get Routing Table does not tell you whether there are endpoints downstream, you have to poll via Get Endpoint ID.

As TMBO, MC1 has allocated a range to MC2, so would be polling that range.

[santoshpuranik]

I was thinking of (1) as well -- could you elaborate a little on the userspace toolset that would be needed here to configure the filter.

Ideally, this would just be adding MCTP support to the existing nft utility. This assumes that the kernel side is implemented similarly to the existing netfilter hooks, but there may be a lot of ip4/ip6 assumptions to unthread as part of doing so.

I am thinking the netfilter would have to be attached on the mctpusbgX netdev

The netfilter hooks do have access to the device in specific tables. This case would be a FORWARD-like table, so you have both the ingress and egress interfaces available for rule matches.

OK, will have to read up on these :)

GetRoutingTable would be the way for MC1 to discover the endpoints downstream to MC2, right?

No, we have a lot of discussion on that in #71. In summary: Get Routing Table does not tell you whether there are endpoints downstream, you have to poll via Get Endpoint ID.

As TMBO, MC1 has allocated a range to MC2, so would be polling that range.

In the picture I have above, MC2 is the one determining EIDs of the downstream devices (and it is not desired to give MC1 that control). I agree that we have to poll via GetEdnpointID to check for existence of the EID, but the problem here is how does MC1 know the list of possible EIDs downstream to MC2? It can still obtain that by a Routing table read, I think/

[jk-ozlabs]

In the picture I have above, MC2 is the one determining EIDs of the downstream devices (and it is not desired to give MC1 that control)

Right, but MC1 is the top-most bus owner, right? In which case, it had allocated MC2's range in the first instance.

MC2 still has full control of how it then uses that range, and the allocation scheme it uses to assign EIDs to device 1 and device 2.

However those allocations must have been originally provided by MC1, which would then be polling.

but the problem here is how does MC1 know the list of possible EIDs downstream to MC2

It was responsible for assigning the entire range, so it knows which EIDs may be downstream.

[santoshpuranik]

It was responsible for assigning the entire range, so it knows which EIDs may be downstream.

I missed your response above. MC2 needs to maintain the EIDs assigned (the exact values) to the MCTP devices downstream. So in a sense, these EIDs belong to its static pool (not received from MC1)

[jk-ozlabs]

So in a sense, these EIDs belong to its static pool (not received from MC1)

That seems irrelevant though - the pool is still allocated by MC1. Unless there is some (platform-specific) mechanism to handle static EID allocations, MC2 cannot hand out arbitrary EIDs without MC1's involvement.

If those are fully-static allocations, then you're not using Allocate Endpoint IDs, and so need to implement your own out-of-band mechanism to communicate (or pre-configure) the EID scheme across your network.

[santoshpuranik]

If those are fully-static allocations, then you're not using Allocate Endpoint IDs, and so need to implement your own out-of-band mechanism to communicate (or pre-configure) the EID scheme across your network.

Yes, they are fully static allocations (at least from the POV of MC1; MC2 is still setting them deterministically). However, the requirement is to let MC1 still communicate with them (for certain MCTP types). I think what you are saying is that such EIDs simply be advertised by other mechanisms (such as documentation) rather than via the routing table?

[jk-ozlabs]

Yep, my reading of DSP0236 is that discovery of such static allocations need to be handled by some external mechanism, and that mechanism is outside of the intended scope of the spec.

[santoshpuranik]

Thanks. I'll see if I can get some clarity from DMTF here.

[jk-ozlabs]

You're going to need an AssignBridgeStatic for this, right?

[santoshpuranik]

You're going to need an AssignBridgeStatic for this, right?

Hmm .. not sure I followed. In the topology above, the intent is to never have MC1 control the exact EIDs being assigned, we don't need any bridge static allocation APIs (we would if we wanted MC1 to hand out deterministic EIDs -- which is still desired for other use cases but not specific to this one)

[jk-ozlabs]

OK, sure thing.