Over the years, many bus protocols have transitioned from a single transaction architecture to a multiple outstanding transaction architecture. The transition from conventional PCI to PCI Express brought split transactions which decoupled the address phase from the data phase. In addition, the on-chip AMBA® AHB® bus was updated to support multiple outstanding addresses with out-of-order responses when the AMBA AXI specification was released.
In the same way, USB 3.1 adds a protocol feature named “multiple INs.” The USB 3.1 host is now allowed to issue a read request to an endpoint on a device and then proceed with other transactions without waiting for the response. After issuing multiple requests to multiple endpoints/devices, the data may return to the host in a different order than it was requested due to arbitration taking place on the devices and the upstream ports of hubs. The host is abstracted away from the exact USB topology in this manner because it cannot predict exactly how fast or in what order the transactions will complete. This can be seen in Figure 4, where the host initiated a transaction to Device 0 first, but it received data packets from Device 1 first.
This change to host behavior also has an effect on USB 3.1 devices. Now, a USB 3.1 device may receive a second ACK TP for a different endpoint before it has a chance to respond to the first ACK TP. A simple USB 3.1 device may choose to continue to handle only one request at a time by responding Not Ready (NRDY) to additional ACK TPs. A more complex USB 3.1 device would take advantage of the multiple INs by servicing the requests separately, possibly returning data to each one, reducing the likelihood that NRDY would need to be used. Figure 5 shows the difference in behavior for a USB 3.1 device that has two or more IN endpoints, depending on whether it supports multiple INs or not.