bizarre performance issue when TSO is enabled

I am writing a de-serialized NDIS driver for a network card, which
supports TSO. My driver works smoothy when TSO is disabled. On windows
2003 enterprise server, netperf can achieve 900+ Mbps. But when TSO is
enabled, the speed drops to 100+ Mbps !

The following is the key functions of the driver.

MiniportHandleInterrupt()
{
NdisAcquireSpinLock(lock);

/* scan the rx ring for pkts */

NdisReleaseSpinLock(lock);
NdisMIndicateReceivePacket();
NdisAcquireSpinLock(lock);

CheckTxCompletion(); -> check tx ring for tx completion and call
NdisMSendComplete
if (txQueue not empty){
send pkts in the txQueue
}
NdisReleaseSpinLock(lock);
}

MiniportMultipleSend()
{
NdisAcquireSpinLock(lock);

queue the pkts into txQueue

CheckTxCompletion(); ****

if (txQueue not empty){
send pkts in the txQueue
}

NdisReleaseSpinLock(lock);
}

If I remove CheckTxCompletion() from MiniportMultipleSend, netperf
thruput comes back to 900+ when tso is enabled. I don't understand why.

What's more strange is that the driver (with the fix mentioned above)
works equally fine in windows 2000 advance server w/ SP4 when tso is
disabled. But when tso is enabled, the thruput drops to 2 Mbps! The cpu
is completely idle during the test.

Has anybody encountered the same issue before? Also, is there any rule
when NdisMSendComplete should be called? Thanks

It seems like you call CheckTxCompletion with
a spinlock held.
If this function calls NdisSendComplete, you call NDIS with
a spinlock held - which can result in a variety of
nasty things, from performance hit to a deadlock.

Regards,
--PA


"" wrote:
> I am writing a de-serialized NDIS driver for a network card, which
> supports TSO. My driver works smoothy when TSO is disabled. On windows
> 2003 enterprise server, netperf can achieve 900+ Mbps. But when TSO is
> enabled, the speed drops to 100+ Mbps !
>
> The following is the key functions of the driver.
>
> MiniportHandleInterrupt()
> {
> NdisAcquireSpinLock(lock);
>
> /* scan the rx ring for pkts */
>
> NdisReleaseSpinLock(lock);
> NdisMIndicateReceivePacket();
> NdisAcquireSpinLock(lock);
>
> CheckTxCompletion(); -> check tx ring for tx completion and call
> NdisMSendComplete
> if (txQueue not empty){
> send pkts in the txQueue
> }
> NdisReleaseSpinLock(lock);
> }
>
> MiniportMultipleSend()
> {
> NdisAcquireSpinLock(lock);
>
> queue the pkts into txQueue
>
> CheckTxCompletion(); ****
>
> if (txQueue not empty){
> send pkts in the txQueue
> }
>
> NdisReleaseSpinLock(lock);
> }
>
> If I remove CheckTxCompletion() from MiniportMultipleSend, netperf
> thruput comes back to 900+ when tso is enabled. I don't understand why.
>
> What's more strange is that the driver (with the fix mentioned above)
> works equally fine in windows 2000 advance server w/ SP4 when tso is
> disabled. But when tso is enabled, the thruput drops to 2 Mbps! The cpu
> is completely idle during the test.
>
> Has anybody encountered the same issue before? Also, is there any rule
> when NdisMSendComplete should be called? Thanks
>
>