View source for Troubleshooting N310/N320 Device Discovery Issues

==Application Note Number==
'''AN-621 '''

==Revision History==
{| class="wikitable"
!Date
!Author
!Details
|-
|style="text-align:center;"| 2020-01-17   
|style="text-align:center;"| Sam Reiter
|style="text-align:center;"| Initial creation
|}

==Introduction==
There are several methods of connecting a USRP N3xx device to a host PC (MGMT Ethernet, Serial, SFP+, and QSFP+), all of which are covered in this guide. We will be using a Linux machine (Ubuntu 19.10) for the following steps, but the troubleshooting steps can be generalized across other operating systems. While this guide is applicable to N300, N310, N320, and N321, several of the sections will be applicable to E3xx devices. In general, it is recommended that you start off by connecting via either MGMT Ethernet or Serial before setting up streaming over the SFP+ or QSFP+ ports. 

It is assumed that UHD has been installed on the host machine. See [https://kb.ettus.com/Building_and_Installing_the_USRP_Open-Source_Toolchain_(UHD_and_GNU_Radio)_on_Linux here] for guidance on building and installing UHD from source.

==Aligning Software Versions on Host and USRP==
On MPM-based devices, like the N3xx, there are several pieces of software that need to be version-aligned to allow for communication. Irrespective of your connectivity issues, start your troubleshooting by completing this list:

<OL>
<LI>Install UHD on the Host Machine (required to interface with N3xx and to download FPGA images)
<UL>
<LI>'''Linux (source): ''' [https://kb.ettus.com/Building_and_Installing_the_USRP_Open-Source_Toolchain_(UHD_and_GNU_Radio)_on_Linux Building and Installing the USRP Open-Source Toolchain (UHD and GNU Radio) on Linux]
<LI>'''Windows (binaries): ''' https://files.ettus.com/binaries/uhd/
</UL>
<LI>Install the same version of UHD on the N3xx
<UL>
<LI>'''Option 1:''' [https://kb.ettus.com/Writing_the_USRP_File_System_Disk_Image_to_a_SD_Card Writing the USRP File System Disk Image to a SD Card]
<LI>'''Option 2:''' [https://kb.ettus.com/USRP_N300/N310/N320/N321_Getting_Started_Guide#Updating_the_file_system_with_Mender Updating the N3xx file system with Mender]
<UL>
<LI> Note: you will have to establish communication over Ethernet/*SFP+ to update the N3xx file system using Mender
</UL>
</UL>
<LI> Install the corresponding version of MPM on the N3xx
<UL>
<LI>This is taken care of by completing the N3xx filesystem update above. Skip if you completed steps 1 and 2.
<LI>'''Update MPM only:''' [https://files.ettus.com/manual/page_usrp_e3xx.html#e3xx_software_dev_mpm_native Compiling MPM Natively]
<UL>
<LI> Note: you will have to establish communication over Ethernet/*SFP+ to compile MPM on the N3xx
</UL>
</UL>
<LI>Update the FPGA on the N3xx
<UL>
<LI> '''UHD Manual:''' [https://files.ettus.com/manual/page_usrp_n3xx.html#n3xx_getting_started_fpga_update Updating the FPGA]
<LI> '''KB Page:''' [https://kb.ettus.com/USRP_N300/N310/N320/N321_Getting_Started_Guide#Updating_the_FPGA_Image Embedded and Network Mode FPGA update instructions] 
<UL>
<LI> Note: This is required for streaming, but you will have to establish communication before you can update the FPGA.
</UL>
</UL>
</OL>

==MGMT Port Ethernet Connection==
KB Page: [https://kb.ettus.com/USRP_N300/N310/N320/N321_Getting_Started_Guide#Connecting_to_the_ARM_via_SSH Connecting to the ARM via SSH]
 
Manual Page: [https://files.ettus.com/manual/page_usrp_n3xx.html#n3xx_getting_started_ssh N3xx Getting Started SSH]

N3xx devices all have an RJ45 port intended for communications with the device. This interface is called the management (mgmt) port and is suitable for SSH, file transfer, and device discovery, but '''not''' CHDR packet streaming. 

<OL>
<LI> <u>Issue: The N3xx doesn't check out a DHCP address from my <code>dnsmasq</code> server</u>
<UL>
<LI> Ensure your USRP is powered on.
<LI> Ensure your host network and dnsmasq server are configured correctly:
<UL>
<LI> Host interface must be configured for a static connection. A subnet of 255.255.255.0 is recommended.
<LI> Make sure you're specifying the appropriate host interface in <code>-i <ETHERNET_ADAPTER_NAME></code>. Available interfaces can be found by running <code>ip a</code> on a host machine.
<LI> Make sure the first 3 octets of your <code>--dhcp-range</code> match the first 3 octets of your host's static IP (assuming you're using the subnet 255.255.255.0).
</UL>
<LI> For network changes to take effect, it's often necessary to reset the network interface. This can be done by physically disconnecting the Ethernet cable from the host's RJ45 port.
<LI> Try using a new Ethernet cable 
<LI> If the N3xx still won't check out an IP, try connecting it to an existing network (company wired network, router RJ45 port, etc.). From here, you can [https://vitux.com/find-devices-connected-to-your-network-with-nmap/ search your network with nmap]. 
<LI> If this device recovery doesn't complete successfully or recovery doesn't bring back the link LED, contact Ettus Support.
</UL>
</OL>

<OL>
<LI> <u>Issue: I am unable to SSH into the device using the MGMT Port</u>
<UL>
<LI> Ensure your USRP is powered on.
<LI> By default (i.e. after burning a new filesystem to the SD card), this port is configured as a DHCP interface. To connect directly from a host computer, you will need to set up a DHCP  
<LI> If a known good SFP+ adapter does not elicit a green LINK LED, you should attempt recovery using the HG image. See [[X300/X310 Device Recovery]].  
<LI> If this device recovery doesn't complete successfully or recovery doesn't bring back the link LED, contact Ettus Support.
</UL>
</OL>

<OL>
<LI> <u>Issue: <code>uhd_find_devices</code> doesn't return</u>
<UL>
<LI> Ensure your USRP is powered on.
<LI> By default (i.e. after burning a new filesystem to the SD card), this port is configured as a DHCP interface. To connect directly from a host computer, you will need to set up a DHCP  
<LI> If a known good SFP+ adapter does not elicit a green LINK LED, you should attempt recovery using the HG image. See [[X300/X310 Device Recovery]].  
<LI> If this device recovery doesn't complete successfully or recovery doesn't bring back the link LED, contact Ettus Support.
</UL>
</OL>

==Serial Connection==
<UL>
*KB Page: [https://kb.ettus.com/USRP_N300/N310/N320/N321_Getting_Started_Guide#Setting_up_a_Serial_Console_Connection Setting up a Serial Console Connection]
*Manual Page: [https://files.ettus.com/manual/page_usrp_n3xx.html#n3xx_getting_started_serial N3xx Getting Started Serial Console]
</UL>

==SFP+ Connection==
The X310 has a pair of SFP+ ports located on the rear panel of the device near the power input. These interfaces are labeled 0 and 1:

<center>
<div><ul>
<li style="display: inline-block;"> [[File:X310_RedBox_Ethernet.jpg|thumb|800px|center|Figure 1: X310 Rear Panel with SFP+ ports emphasized.]] </li>
</ul></div>
</center>


It is possible to connect to the X310 over 1 Gigabit Ethernet via an SFP+ to RJ45 adapter or a 10 Gigabit Ethernet capable SFP+ cable. Whether or not your X3xx device is capable of using 1GbE is dependent on the FPGA image being used. With the HG image port 0 is 1GbE and port 1 is 10GbE. Using the XG image enabled dual 10GbE links (ports 0 and 1). See the [https://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_getting_started_fpga_update UHD Manual] for instructions regarding updating the FPGA image of your device. 

===1 Gigabit Ethernet (1GbE)===

This protocol is supported only on X3xx SFP+ port 0 when using the HG FPGA image. If both of these conditions are met, you will see the green LINK LED illuminate when a working SFP+ adapter is inserted into the powered-on X310. On X3xx devices, this LED should light up regardless of successful connection to host when a working SFP+ adapter is inserted.

<center>
<div><ul>
<li style="display: inline-block;"> [[File:X310_LINK_LED.jpg|thumb|200px|center|Figure 2: X310 SFP+ ports with link and data LEDs.]] </li>
</ul></div>
</center>

<OL>
<LI> <u>Issue: Ethernet LINK LED (Port 0) does not illuminate</u>
<UL>
<LI> Ensure your USRP is powered on.
<LI> If the green LINK LED does not illuminate in port 0, you should try a different (known working) SFP+ adapter.  
<LI> If a known good SFP+ adapter does not elicit a green LINK LED, you should attempt recovery using the HG image. See [[X300/X310 Device Recovery]].  
<LI> If this device recovery doesn't complete successfully or recovery doesn't bring back the link LED, contact Ettus Support.
</UL>
</OL>

With the HG FPGA image and a good SFP+ adapter confirmed, the device can be connected to a host PC directly from the computer's RJ45 ethernet port. The host machine should be able to ping the radio using a "ping <X310 Port0 IP address>" command.

<OL START="2">
<LI> <u>Issue: Host PC is unable to ping the USRP</u>
<UL>
<LI> Ensure your USRP is powered on. Ensure your host network settings are correct. See [https://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_setup_network_host_interface USRP Manual: Setup the host interface]. Note that many systems require a user to unplug / replug the ethernet interface for configuration changes to take effect.  
<LI> Ensure host is connected directly to the X310. Eliminate all routers and switches in between USRP and host machine. (**Note on this at the end of this list)
<LI> Use uhd_find_devices to discover the IP of USRP devices on the same subnet as the host. If discovered, ping the IP address returned by uhd_find_devices. 
<LI> Use an interface probe utility (such as [https://www.wireshark.org/ Wireshark]) to "sniff" the ethernet port being used. The X310 should emit a gratuitous ARP request every 15 seconds. When this request reveals the X310’s IP, you can adjust your host IP and subnet configuration and ping the radio.
<LI> Replace your RJ45 cable and repeat steps the above steps for this issue.
<LI> Some communication issues have been observed with specific Linux distros (Kubuntu is one example) '''requiring''' a network switch between Host and USRP - adding an intermediate switch is worth exploring in the event other steps in this were unsuccessful.
<LI> If you are still unable to ping your X310, contact Ettus support.
</UL>
</OL>

If you're able to ping the device, then you have successfully connected to it. At this point, it is recommended to run uhd_usrp_probe to pull device information. If your FPGA image is not consistent with your version of UHD on the host side, you may get errors or warnings regarding FPGA version numbers. See the [https://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_getting_started_fpga_update USRP Manual: Updating the FPGA section].

===10 Gigabit Ethernet (10GbE)===

This protocol is supported on USRP Ethernet port 1 when using the HG FPGA image. When using the XG image, 10 Gigabit Ethernet is supported on ports 0 and 1. Unlike the SFP+ adapter's behavior the section “1 Gigabit Ethernet” section above, you should only expect to see the LINK LED turn green for a 10 Gigabit port upon successful connection to a host machine.

<OL>
<LI> <u>Issue: Ethernet LINK LEDs do not illuminate when connected to host machine</u>
<UL>
<LI> Ensure your USRP is powered on.
<LI> Ensure the SFP+ cable is plugged into port 1 of the USRP. This is a 10 Gig interface on both the HG and XG images.
<LI> If your host machine is using a PCIe adapter for SFP+ interfaces, ensure it was enumerated correctly by your operating system. You may need to reinstall drivers or try a different PCIe port.  
<LI> Check that the SFP+ interface on your host machine is configured to support a 10 Gigabit link. If your NIC supports dual 10 GbE links, you can put it in loopback with your SFP+ cable and confirm that both ports recognize a 10GbE connection:

<center>
<div><ul>
<li style="display: inline-block;"> [[File:Ethernet_toggle.png|thumb|600px|center|Figure 3: Ubuntu 18.04 Network Manager with 2 10GbE interfaces in loopback]] </li>
</ul></div>
</center>

<LI> Swap your cable for a different (known good SFP+ to SFP+ cable).  
<LI> If a known good SFP+ cable connected to a known working SFP+ port does not elicit a green LINK LED, you should attempt device recovery with the XG image. See [[X300/X310 Device Recovery]]. With the XG image, try the above steps on Port 0 of your X310. 
<LI> If this device recovery doesn't complete successfully or recovery doesn't bring back the link LED, contact Ettus Support.
</UL>
</OL>

Once the device has a successful 10GbE connection to the host, the host machine should be able to ping the radio using a "ping <X310 Port IP address>" command.

<OL START="2">
<LI> <u>Issue: Host PC is unable to ping the USRP</u>
<UL>
<LI> Ensure your USRP is powered on
<LI> Double check that you're pinging the correct IP address. Each port will have a unique IP depending on whether you're using the HG or XG image. Note that the values in the figure shown below may have been changed from their default IP settings:


<center>
<div><ul>
<li style="display: inline-block;"> [[File:IP_Addresses.png|thumb|700px|center|Figure 4: Default USRP network configuration reported by uhd_usrp_probe]] </li>
</ul></div>
</center>


<LI> Ensure your host network settings are correct. See [https://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_setup_network_host_interface USRP Manual: Setup the host interface]. Note that many systems require a user to unplug / replug the ethernet interface for configuration changes to take effect.  
<LI> Ensure host is connected directly to the USRP. Eliminate all routers and switches in between USRP and host machine.
<LI> Use uhd_find_devices to discover the IP of USRP devices on the same subnet as the host. If discovered, ping the IP address returned by uhd_find_devices.
<LI> Use an interface probe utility (such as [https://www.wireshark.org/ Wireshark]) to "sniff" the ethernet port being used. The X310 should emit a gratuitous ARP request every 15 seconds. When this request reveals the X310’s IP, you can adjust your host IP and subnet configuration and ping the radio.
<LI> Replace your SFP+ cable and repeat the above steps of this section.
<LI> If you are still unable to ping your device, contact Ettus Support.
</UL>
</OL>

If you're able to ping the device, then you have successfully connected to it. At this point, it is recommended to run uhd_usrp_probe to pull device information. If your FPGA image is not consistent with your version of UHD on the host side, you may get errors or warnings regarding FPGA version numbers. See the [https://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_getting_started_fpga_update USRP Manual: Updating the FPGA section].

==QSFP+ Connection==

==Miscellaneous Notes==
Early runs of the N300 and N310 had a firmware bug that resulted in intermittent issues with power button response. This can often be bypassed by issuing the command "powerbtn" over the serial console to the ARM. Reach out to support@ettus.com for instructions on downloading and installing the patch.

==Additional Resources==
<UL>
<LI> [https://files.ettus.com/manual/page_usrp_x3x0.html USRP Hardware Driver and USRP Manual: USRP X3x0 Series]
<LI> [https://kb.ettus.com/X300/X310 Ettus Knowledge Base: X300/X310]
<LI> [https://kb.ettus.com/Verifying_the_Operation_of_the_USRP_Using_UHD_and_GNU_Radio Verifying the Operation of the USRP Using UHD and GNU Radio]
</UL>


[[Category:Application Notes]]