USRP Host Performance Tuning Tips and Tricks

From Ettus Knowledge Base
Jump to: navigation, search

Application Note Number

AN-088

Overview

This application note provides various tips and tricks for tuning your host computer for best performance when working with USRP devices.

CPU Governor

NOTE: use of cpufrequtils noted below should be limited to Linux kernel 5. The utility cpupower should be used in Linux kernel 6 and newer.

Ensure your CPU governor is set to performance. This can be done with the Linux utility cpufrequtils.

Install cpufrequtils with the command below: 

   sudo apt install cpufrequtils

You can then set the CPU governor to performance per core by issuing the command: 

   sudo cpufreq-set -c $core_number -g performance

To set the CPU governor to performance for all cores: 

   for ((i=0;i<$(nproc --all);i++)); do sudo cpufreq-set -c $i -r -g performance; done

You can then verify that the CPU governor has been set by running the command: 

   cpufreq-info

Thread Priority Scheduling

When UHD spawns a new thread, it may try to boost the thread's scheduling priority. If setting the new priority fails, the UHD software prints a warning to the console, as shown below. This warning is harmless; it simply means that the thread will retain a normal or default scheduling priority. 

UHD Warning:
    Unable to set the thread priority. Performance may be negatively affected.
    Please see the general application notes in the manual for instructions.
    EnvironmentError: OSError: error in pthread_setschedparam

To address this issue, non-privileged (non-root) users need to be given special permission to change the scheduling priority. This can be enabled by creating a group usrp, adding your user to it, and then appending the line @usrp - rtprio 99 to the file /etc/security/limits.conf. 

   sudo groupadd usrp
   sudo usermod -aG usrp $USER

Then add the line below to end of the file /etc/security/limits.conf: 

   @usrp - rtprio  99

You must log out and log back into the account for the settings to take effect. In most Linux distributions, a list of groups and group members can be found in the /etc/group file.

There is further documentation about this in the User Manual at the link below.


Adjust Network Buffers

This applies to USRP devices connected via 10 GbE or 100 GbE Ethernet: N2xx, N3xx, X3xx, E320, X4xx

Note that these settings will not persist across a reboot. 

   sudo sysctl -w net.core.wmem_max=33554432
   sudo sysctl -w net.core.rmem_max=33554432
   sudo sysctl -w net.core.wmem_default=33554432
   sudo sysctl -w net.core.rmem_default=33554432

To make these settings persistent, set or add the following in /etc/sysctl.conf: 

   net.core.wmem_max=33554432
   net.core.rmem_max=33554432
   net.core.wmem_default=33554432
   net.core.rmem_default=33554432

NOTE: You may choose to use a larger setting such as 250000000 to guarantee these settings work for any USRP. Larger numbers require memory allocation but with modern hardware this memory is generally available with plenty of spare.

Adjust Ethernet MTU

This applies to Ethernet connected USRPs: N2xx, N3xx, X3xx, E31x, E320, X4xx

For 1 GbE links, the MTU should be set to 1500.

For 10 GbE and 100 GbE links, the MTU should be set to 9000.

It is important to set the value and not leave it in the default: automatic

Increasing Ring Buffers

This applies to Ethernet connected USRPs using a 10 GbE or 100 GbE interface: N2xx, N3xx, X3xx, E320, X4xx

Increasing the Ring Buffers on the NIC may help prevent flow control errors at higher rates. For example: 

   sudo ethtool -G <interface> tx 4096 rx 4096

To see the NIC limits, use 

   sudo ethtool -g <interface>

DPDK

DPDK is supported on USRPs with 10 Gbe (SFP+) and 100 GbE (QSFP28) links. DPDK replaces the traditional Linux networking stack and NIC driver with a low overhead user-land based driver. See the DPDK AppNote for specifics.

Disable Hyper-threading

In some applications which require the highest possible CPU performance per core, disabling hyper-threading can provide roughly a 10% increase in core performance, at the cost of having fewer core threads. Hyper-threading is disabled within the BIOS and how to do this varies by motherboard manufacturer. With other techniques listed here, disabling hyper-threading should only be done as a last resort to eek absolute maximum performance from the CPU.

Disable KPTI Protections for Spectre/Meltdown

In some cases, disabling the KPTI protections for the Linux Kernel can increase performance by 10-15%. It is important to note the ramification making this modification can have. This modification is only recommended for systems that absolutely require the best performance and are not connected to the internet.

Disabling KPTI protections can be done by adding the lines below to your /etc/default/grub file at GRUB_CMDLINE_LINUX_DEFAULT="".

For Linux kernel 5: 

   pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier

For Linux kernel 6 or higher: 

   mitigations=off

NOTE: After saving the GRUB_CONFIG file, execute sudo update-grub and then reboot the computer for the changes to take effect.

Retrieved from "https://kb.ettus.com/index.php?title=USRP_Host_Performance_Tuning_Tips_and_Tricks&oldid=7226"