XBee and USB to Serial FTDI latency and dropped packets

You would think that the USB board provided with the XBee dev kit right would work perfectly right out of the box. Unfortunately, you may experience problems depending on your application.

The Problem - Buffering

USB works quite different than RS232, or any serial type communication. A USB client is polled by the host. This means that the client must sit on any data it has until the host asks for it. In a USB to serial converter, the FTDI chip buffers data it receives until either the buffer is full, or until an idle timer triggers the buffer to flush because no data has come on the serial port for a configured amount of time.

The Ugly Side of Buffering

Buffering the data can cause two things to happen. If the data comes in at just the right speed, the buffer will slowly fill, but the timer will not time out. The default timeout for the FTDI driver is 16ms. It takes 62 bytes coming from the serial port to reset the timer. If the FTDI chip receives 62 bytes every 16ms, a timeout will never occur, and the USB block request buffer will slowly fill. The FTDI chip will then only send data out once the USB block request is full. You can figure out the time it will take the buffer to fill with the following equation, given the default block request buffer size is 4096 bytes.

time = 16ms*(size of ftdi buffer)/(62 serial bytes + 2 status bytes)

Worst case,

16ms*4096B/64B = 1.024 seconds

This 4k of data will come in bursts every 1 second. In computing time, this is eternity. Additionally, when the data does come, it comes in bursts of 4kB and at full blast. There is no delay between bytes. If the program or hardware receiving the data from the FTDI over the virtual serial port doesn't service the data fast enough, data will be lost and packets dropped.

The Dropped Packet Solution - Flow Control

To solve the problem of lost data, use flow control. Flow control might not be required on a physical RS232 interface with the same throughput and delays, but on the virtual port it is almost requisite. Enabling flow control is a simple thing to do, especially on the USB board that comes with the XBee dev kit. Any device connected to the FTDI chip should use flow control of some sort.

The Latency Solutions - Block Request Size and Latency Timer

There are 2 ways to fix the latency issues. Fist, you can change the block request size from 4kB down to a smaller value. The second method is to decrease the latency timer value from 16ms to a smaller value. According to FTDI, the preferred method is to change the block request size. In reality a balance of the two methods will yield the best performance per application.

To change the block request size and latency timer settings in windows, open the device manager. In the device manager, find the virtual serial port under 'Ports'. Open up the properties. Under 'Advanced' options on the 'Port Settings' tab, you can change both block request size and latency timer settings.

More information can be found here.