Question on out-of-order delivery inside the replayer

[eunju1010]

Hi, thanks for your work! I'm using Leoreplayer using the provided traces and find that packets are arriving out of order.

I'm sending a UDP/RTP packet from the outer(streamer) to the inner(viewer) shell.
When I check, the viewer is sending too many NACKs even though there is low packet loss. I found out that this is happening due to the out-of-order arrival of seq on the viewer side.

These are outer capture files.

frame.time_epoch,ip.src,ip.dst,udp.srcport,udp.dstport,udp.length,rtp.ssrc,rtp.p_type,rtp.seq,rtp.timestamp,rtp.marker
1779827370.769031000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,27997,34891439,0
1779827370.769091000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,27998,34891439,0
1779827370.769122000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,27999,34891439,0
1779827370.769155000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28000,34891439,0
1779827370.769185000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28001,34891439,0
1779827370.769218000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28002,34891439,0
1779827370.769245000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28003,34891439,0
1779827370.769290000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28004,34891439,0
1779827370.769326000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28005,34891439,0
1779827370.769360000,100.64.0.1,100.64.0.2,45605,50926,1444,0x3b5e9789,101,28006,34891439,0

And these are inner capture files.

frame.time_epoch,ip.src,ip.dst,udp.srcport,udp.dstport,udp.length,rtp.ssrc,rtp.p_type,rtp.seq,rtp.timestamp,rtp.marker
1779827370.781540000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,27998,34891439,0
1779827370.781546000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28002,34891439,0
1779827370.781549000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28010,34891439,0
1779827370.781551000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28009,34891439,0
1779827370.781554000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28001,34891439,0
1779827370.781556000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28008,34891439,0
1779827370.781557000,100.64.0.1,100.64.0.8,45605,50926,1160,0x3b5e9789,101,28021,34891439,1
1779827370.781559000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28007,34891439,0
1779827370.781561000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28020,34891439,0
1779827370.781563000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,28019,34891439,0
1779827370.781565000,100.64.0.1,100.64.0.8,45605,50926,1444,0x3b5e9789,101,27997,34891439,0

You can see that even though outer's rtp.seq are sequential, inner's rtp.seq are out-of-order.

The run.sh starts the outer.sh and inner.sh

echo "[run.sh] Launching outer.sh (streamer side) ..."
bash "$DIR/outer.sh" "$RUNNING_TIME" "$SERVER_DIR" &

mm-delay "$DELAY_INTERVAL" "$TRACE_DELAY_PATH" \
  mm-loss uplink "$UPLINK_LOSS_RATE" \
  mm-loss downlink "$DOWNLINK_LOSS_RATE" \
  mm-link "$TRACE_BW_UPLINK" "$TRACE_BW_DOWNLINK" \
    --uplink-queue drophead   --uplink-queue-args   packets=$UPLINK_PACKET_LENGTH \
    --downlink-queue drophead --downlink-queue-args packets=$DOWNLINK_PACKET_LENGTH \
  bash "$DIR/inner.sh" "$RUNNING_TIME" "$VIEWER_DIR"

Is this kind of out-of-delivery common, as the replayer applies latency independently per packet? Or am I running the replayer in the wrong way?

Thanks for your help in advance!