Difference between revisions of "Getting Started with RFNoC Development"

From Ettus Knowledge Base
Jump to: navigation, search
(→‎Adding custom blocks)
(Emphasize that this application is deprecated)
 
(121 intermediate revisions by 8 users not shown)
Line 1: Line 1:
−
= NOT FINISHED DONT READ! =
+
==Application Note Number==
  
−
= Overview =
+
'''AN-823'''
 +
<!-- Internal use only: please do keep this updated!
 +
==Revision History==
 +
{| class="wikitable"
 +
!Date
 +
!Author
 +
!Details
 +
|-
 +
|style="text-align:center;"| 2016-07-12
 +
|style="text-align:center;"| Martin Braun<br> Nicolas Cuervo
 +
|style="text-align:center;"| Initial creation
  
−
This application note gives a brief introduction into the steps required to start developing RFNoC blocks on your computer.
 
−
First, you need to make sure that you have all the tools required to do RFNoC development.
 
−
Then, we will set up a development sandbox in which you can do your RFNoC development.
 
−
Finally, we will run through the basic steps required to actually write custom RFNoC blocks.
 
  
 +
|-
 +
|style="text-align:center;"| 2017-01-10
 +
|style="text-align:center;"| Team
 +
|style="text-align:center;"| Added “Digital Gain” example
 +
|-
  
−
== Prerequisites ==
+
|-
 +
|style="text-align:center;"| 2017-05-08
 +
|style="text-align:center;"| Jose Loera
 +
|style="text-align:center;"| Updated example code. Update to Testbench section.
 +
|-
  
−
For any of this to work, you need to make sure you have all the tools to do both software and FPGA development.
+
|-
−
Confirm you have the following tools installed:
+
|style="text-align:center;"| 2017-08-26
 +
|style="text-align:center;"| Jose Loera
 +
|style="text-align:center;"| Updated following sections: '''Abstract'''(This AN is specific to USRP X300/X310), '''Using a graphical interface'''(updated GUI image with newest version and the explanation section), '''Testing out the custom block'''(Updated GRC image that has correct Sampling Rate for RFNoC:Radio block).
 +
|-
  
−
* Xilinx Vivado. The specific version depends on the branch and state of the FPGA code and is very important. Make sure to install the correct version.
+
|-
−
* Software build tools. If you can compile UHD from source, you have all the necessary components (If you're using PyBOMBS
+
|style="text-align:center;"| 2017-09-07
 +
|style="text-align:center;"| Jose Loera
 +
|style="text-align:center;"| Added link to Video that follows this App Note in the Resources section. Also [https://youtube.com/watch?v=j-EfyPVpaJ8 here]
 +
|-
  
−
Please make sure these tools are available before proceeding to the next step.
+
|-
 +
|style="text-align:center;"| 2019-10-24
 +
|style="text-align:center;"| Michael Dickens
 +
|style="text-align:center;"| Fixed list of USRPs that this AN is applicable to: all current 3rd generation USRP hardware.
 +
|-
  
 +
|}
 +
-->
  
−
== Creating a development environment ==
+
==Abstract==
 +
'''Note: This application note is DEPRECATED. It applies only to using RFNoC in UHD 3.x. The latest Getting Started in RFNoC application note is available here: [[Getting Started with RFNoC in UHD 4.0]].'''
  
−
In the following, we will assume that GNU Radio is used as a helper tool for RFNoC development. Note that GNU Radio is by no means required to use or develop RFNoC, but it makes it a great deal easier. So in order to get your development environment set up, you will need the following software packages:
+
This application note guides a user through basic information on the RFNoC architecture, installing necessary software to develop custom RFNoC blocks, also called Computation Engines (CE), and walks through the steps of creating a custom RFNoC block using an example. RFNoC is currently supported on any 3rd generation USRP hardware, currently: E310/E312, E320, N300/N310/N320/N321, and X300/X310.  '''However''', this document primarily covers using RFNoC for the USRP X300/X310 and E310/E312. Using RFNoC with the other USRPs is similar to that documented herein.
  
−
- UHD
+
==Overview==
−
- GNU Radio
+
First sections deal with installing tools and validating correct tool installation in order to do RFNoC development. Later sections deal with creating a custom RFNoC block, using the built-in testbench architecture, building an FPGA image with the custom block and finally testing out the new block within GNU Radio.
−
- gr-ettus
+
  
−
The cleanest way to set this up is to install everything into a dedicated directory. PyBOMBS is the simplest way to do this.
+
==Licensing==
−
If you haven't already downloaded and installed PyBOMBS, you can do so by running the following commands:
+
The RFNoC code base is open source, including code that executes on the host, as well as code targeted to the USRP hardware (FPGA and microcontroller firmware). RFNoC is available under the open-source GNU Lesser General Public License (LGPL). For more information on our licensing policy, please contact [mailto:info@ettus.com info@ettus.com].
  
 +
==Prerequisites==
 +
RFNoC is only supported on 3rd generation USRP hardware as noted in the Abstract.
 +
 +
In order to build custom USRP FPGA images and RFNoC blocks the following hardware and software are needed.
 +
 +
* '''Ubuntu 14.04.5 or 16.04.1 (preferred):''' Currently PyBOMBS (which can be used to install the ''Software build tools''), works most reliably in Ubuntu, and thus, we recommend using this distribution. Also, a majority of the scripts used during the build process are Linux (Ubuntu) specific. A PC with multiple cores and 8GB+ of RAM is recommended.
 +
 +
* '''Xilinx Vivado tools (version 2017.4):''' The specific version depends on the branch and state of the FPGA code. The default install location is <code>/opt/Xilinx/Vivado</code>. Once all of the Software build tools are installed the specific version for the downloaded code can be found in the <code>setupenv.sh</code> file located in the <code>{USER_PREFIX}/src/uhd-fpga/usrp3/top/{DEVICE}</code> directory. Further information can be found [http://files.ettus.com/manual/md_usrp3_build_instructions.html here].
 +
 +
* '''Software build tools:''' If UHD can be or has been compiled from source on the development PC then all the necessary software build components are present (PyBOMBS can be used to set all this up and instructions on how to do so are given in a following step).
 +
 +
* Any 3rd generation USRP hardware as noted in the Abstract.
 +
 +
'''NOTE:'''
 +
* The edition of Xilinx Vivado that is required will depend on which USRP device is being used.
 +
** X3xx series devices: Design Edition or System Edition.
 +
** E3xx series devices: Design Edition, System Edition, or the free WebPack Edition.
 +
* Other operating systems can be used, but the exact steps on how to proceed are not given in this Application Note.
 +
* In some Linux distributions (e.g. Ubuntu) <code>dash</code> is set as default shell, which may cause some issues. It is recommended to set the shell to <code>bash</code> by running the following commands in the terminal. Choose <code><No></code> when prompted by the first command and the second command will validate the that bash will be used.
 +
 +
    $ sudo dpkg-reconfigure dash
 +
    $ ll /bin/sh
 +
 +
==Creating a development environment==
 +
While this Application Note goes through the process of integrating GNU Radio into the RFNoC development flow, it is by no means required to use or develop within the RFNoC framework, but it makes it a great deal easier to use a framework on top of RFNoC for aspects such as visualization and other features. GNU Radio is freely available and more information about it can be found [http://gnuradio.org/ here].
 +
 +
The following software packages are required in order to setup a development environment/sandbox:
 +
 +
* UHD
 +
* GNU Radio
 +
* gr-ettus
 +
 +
===Create development environment using PyBOMBS===
 +
The cleanest way to set this up is to install everything into a dedicated directory. [https://github.com/gnuradio/pybombs PyBOMBS] is the simplest way to do this. If not already installed, PyBOMBS can be setup with the following commands:
 +
 +
    $ sudo apt-get install git
 +
    $ sudo apt-get install python-setuptools python-dev python-pip build-essential
 +
   
 
     $ sudo pip install git+https://github.com/gnuradio/pybombs.git
 
     $ sudo pip install git+https://github.com/gnuradio/pybombs.git
 
     $ pybombs recipes add gr-recipes git+https://github.com/gnuradio/gr-recipes.git
 
     $ pybombs recipes add gr-recipes git+https://github.com/gnuradio/gr-recipes.git
 
     $ pybombs recipes add ettus git+https://github.com/EttusResearch/ettus-pybombs.git
 
     $ pybombs recipes add ettus git+https://github.com/EttusResearch/ettus-pybombs.git
  
−
From here, the easiest way to set up a PyBOMBS development environment is to run the following command:
+
These commands will do the following:
 +
* Install <code>Git</code>
 +
* Install <code>pip</code> and other Python dependencies
 +
* Install the latest <code>PyBOMBS</code> from its Git repository
 +
* Add the <code>gr-recipes</code> recipes which are used to install GNU Radio specific software
 +
* Add the <code>ettus</code> recipes which are used to install Ettus Research specific software
 +
 
 +
From here, PyBOMBS can be used to setup and install the development environment/sandbox by running the following command:
  
−
     $ pybombs prefix init ~/prefix/rfnoc -R rfnoc [-a alias]
+
     $ pybombs prefix init ~/rfnoc -R rfnoc -a rfnoc
  
 
This will do the following:
 
This will do the following:
  
−
- Create a directory in ~/prefix/rfnoc
+
* Create a directory in the user’s home directory called <code>rfnoc</code> (any valid directory name will work)
−
- Clone UHD, FPGA, GNU Radio, and gr-ettus sources into that directory
+
−
- Compile and install all the software
+
  
−
Then, enable the prefix:
+
* Give the prefix an alias of <code>rfnoc</code> ( <code>[-a alias]</code>, e.g. <code>–a rfnoc</code> ), which would be the name given to this path. This name will be used in further steps that use PyBOMBS. When creating the first prefix and omitting the alias, the prefix will be setup as the default.
  
−
     $ cd ~/prefix/rfnoc
+
* Use the <code>rfnoc</code> prefix recipe ( as opposed to a package recipe like <code>gqrx</code> ) to clone UHD, FPGA, GNU Radio, and gr-ettus sources into the <code>~/rfnoc</code> directory as well as compile and install all the software
 +
 
 +
'''NOTE:''' A user can specify how many cores are used by builds when using PyBOMBS. The default is set to 4. For example, this will set the number of cores used to 3:
 +
 
 +
     $ pybombs config makewidth 3
 +
 
 +
The value will be written into a configuration file and then applied to subsequent PyBOMBS commands. This value can temporarily be overridden for a specific build by specifying the <code>--config makewidth=X</code> argument, where “<code>X</code>” is an integer number. If the user only has 4 cores it is recommend to use this argument in the pybombs command to limit the number of cores to <4 (e.g. 3) so that the computer stays responsive. Following are 2 examples, one using less cores and the other using more cores:
 +
 
 +
    $ pybombs --config makewidth=3 prefix init ~/rfnoc -R rfnoc -a rfnoc
 +
    $ pybombs --config makewidth=7 prefix init ~/rfnoc -R rfnoc -a rfnoc
 +
 
 +
Then, it is necessary to setup the PyBOMBS environment, so that the system/terminal session will have the environmental variables pointing to this newly created prefix, which is done with the following commands:
 +
 
 +
    $ cd ~/rfnoc
 
     $ source ./setup_env.sh
 
     $ source ./setup_env.sh
  
−
PyBOMBS is by no means required to do any of these steps. If you wish to install the software manually, follow the individual install notes for GNU Radio, UHD and gr-ettus and make sure they are reachable by linkers and compilers.
+
Once the previous command is run, this terminal session will have access to the environmental variables that allow the complete use of the set of software that was just installed with PyBOMBS. If access to the software is needed in other terminals the same command must be run within them.
  
−
To manually download the software, use these git clone commands, which will select the correct branches:
+
'''NOTE:''' Throughout the rest of this document the term <code>{USER_PREFIX}</code> will used at the beginning of different directories. For example, <code>{USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts</code> is a directory that contains useful scripts for compiling. The term <code>{USER_PREFIX}</code> is used to denote the folders that precede the <code>/src</code> directory. Examples of what <code>{USER_PREFIX}</code> could be: <code>/home/user/rfnoc</code> or <code>/home/user/myDevfolder/</code>. On many Linux environments using <code>~/</code> at the beginning of the target directory path is equivalent to the user’s home directory.( i.e <code>~/</code> is equal to <code>/home/user/</code>). So <code>{USER_PREFIX}</code> could also look like <code>~/rfnoc</code>  or <code>~/myDevfolder/</code>.
  
−
    $ git clone --recursive -b rfnoc-radio-redo https://github.com/EttusResearch/uhd.git
+
===Create the development environment manually===
−
    $ git clone -b maint https://github.com/gnuradio/gnuradio.git # master branch is also fine
+
As an alternative to using PyBOMBS, manually installing and configuring the software is done by following the individual install notes for [http://gnuradio.org/redmine/projects/gnuradio/wiki/InstallingGRFromSource GNU Radio], [https://files.ettus.com/manual/page_build_guide.html UHD] and [https://github.com/EttusResearch/gr-ettus gr-ettus] and by making sure they are reachable by linkers and compilers.
−
    $ git clone -b radio-redo https://github.com/EttusResearch/gr-ettus.git
+
  
−
If you have already UHD, GNU Radio and/or gr-ettus installed, it would be sufficient to checkout to the branches mentioned and pull them. Thereafter, build each of the repositories (first UHD, then GNU Radio, then gr-ettus).
+
'''NOTE:''' The Application Note found [https://kb.ettus.com/Building_and_Installing_the_USRP_Open-Source_Toolchain_(UHD_and_GNU_Radio)_on_Linux here] goes through the process of manually installing UHD and GNU Radio on Linux platforms.
−
== RFNoC Framework ==
+
−
Before explaining the tool utilization, it is better to have clarity on what is being done underneath your design. For this, lets review briefly how the RFNoC Framework structure is set up in our systems. The colourful image down here shows the basic structure of the RFNoC Stack.
+
  
 +
To manually download the software, use these <code>git clone</code> commands, which will select the correct branches:
  
−
     [[File:rfnoc_stack.png|RFNoC Stack|600px|left|RFNoC Stack|link=https://kb.ettus.com/images/3/34/rfnoc_stack.png]]
+
     $ git clone --recursive -b rfnoc-devel https://github.com/EttusResearch/uhd.git
 +
    $ git clone --recursive -b maint https://github.com/gnuradio/gnuradio.git # master branch is also fine instead of maint
 +
    $ git clone -b master https://github.com/EttusResearch/gr-ettus.git
 +
    $ git clone -b rfnoc-devel https://github.com/EttusResearch/fpga.git
  
−
From the stack there are clearly 5 different elements which a RFNoC developer has to take into account when creating an out-of-three block. How this elements interact with each other is explained in more detail at [2][3][4]. However, from here we could easily infer that there are, at least, 3 questions that our Out-of-three design had to answer until this day:
+
If UHD, GNU Radio and/or gr-ettus are already installed, it would be sufficient to checkout the branches mentioned and update them them (<code>git pull</code>). Thereafter, rebuild each of the repositories (rebuild order: UHD, GNU Radio, gr-ettus).
  
−
;1. Does it have a complete FPGA Integration?
+
===Verify Environment===
−
:A verilog file that describes the hardware implementation is what allows us to take advantage of the powerful FPGA that is being shipped with the current and next gen USRPs. For the current stage of development, the .v file has to not only implement the custom desired functionality but also a efficient interaction with the AXI bus and other FPGA blocks. In addition, it the file has to be located in an specific location so that the building process can find it and use it as a resource for the IP generation.  
+
Running the command “<code>uhd_config_info</code>” with the “<code>--version</code>” flag will verify that the installation has been completed successfully.
  
−
;2. Does it have a block declaration/control?
+
'''NOTE:''' The version string output from this command may differ, however it should be similar to the output below.
−
:Now that the design has the hardware implementation in verilog, we would like to take the samples out of the USRP and receive them at our host. The block has to have a clear declaration which defines, at least, its inputs and outputs (which are self explanatory) and the NoC-ID, which is the method of identification between the HW part of our design and the SW part. Now, we might add some control over the samples that are being streamed into the block, and for that we have two options that depend on the complexity of the control that we want to apply: if as developers we can add control using NoC-Script, that could easily be added into the XML file. But if we need more exhaustive control and multiple data stream treatment, we might want to add more coding into it. Block controllers can be added as C++ in the UHD host, carefully generating the headers and adding them in the correct include folder and in the same manner, develop our control and add it in the library folder.  
+
  
−
;3. Does it have GNURadio integration?
+
    $ uhd_config_info --version
−
:GNURadio is where the application runs, and where more control over the host is applied, by adding block-specific settings, such as sample rate or bandwidths. In addition, is where we have the our block ready to connect it to other several blocks (which can be also RFNoC blocks). That means that we are almost done with our design. In order to do so, we have to add a new XML file that allows us to bind our design with GNURadio, and where we can edit the type on our I/Os, add variables for posterior signal processing, and a bunch of other design-specific parameters. In the same XML file we could add, just as we could have done at the block controllers, more block control by the means of NoC-Script. If we choose to do that, we could point our design to a "generic" implementation to control our GRC block. If we, otherwise, need even more control, we could choose to generate block code, that afterwards has to be located at gr-ettus: the headers as an include, the implementations as libraries.
+
    linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-161- g83150fdd
 +
   
 +
    4.0.0.rfnoc-devel-161-g83150fdd
  
 +
===Testing the default FPGA image and building from existing blocks===
  
−
So, you may think that this is just normal protocol of the RFNoC OOT block generation method. But, in order to answer this three questions with your OOT design, you have to take care of correct FPGA interfacing, correct API instantiation and right includes and correct location where to put your files... while you also put effort to the development of your custom design! Here at Ettus, we think that you should put all your effort in your custom design and let the '''RFNoC modtool''' do the dirty work.
+
It is recommended to spend a moment looking at the Ettus Research default image, which is pre-built with a set of RFNoC blocks, as well as building a custom image with a unique set of pre-built RFNoC blocks. To get the default image(s), run the following command:
  
−
== RFNoC Modtool ==
+
    $ uhd_images_downloader
  
−
In order to minimize the effort needed to implement a new RFNoC module, this gr_modtool-based tool provides the basic structure of the necessary files needed for the user to start with the development of his/her custom design right away. As it is gr_modtool-based, the manual found at [1] is also a great reference for this tool. Here, we discuss the features specific for the RFNoC modtool.
+
Ettus Research will be updating the default image(s) occasionally, and <code>uhd_images_downloader</code> can be run anytime after running <code>git pull</code> and re-installing to pull the most current images. Images are stored in the <code>{USER_PREFIX}/share/uhd/images</code> directory.
  
−
With the use of RFNoC ModTool, you can skip the struggle of locating the files of your design at different locations that depend (until now) on the location of your UHD and gr-ettus installation paths. Instead, you focus on your implementation and the tool takes care of boilerplate code and linking.
+
The following images have the corresponding RFNoC blocks (Computation Engines):
  
−
With the use of this tool, the questions stated in the last section would result in something like:
+
{| class="wikitable"
−
;1. Does it have a complete FPGA Integration?
+
!Image Name
−
: If not, start your design in the pregenerated verilog file, which also provides most of the standard AXI interface already done.
+
!Included Blocks
 +
|-
  
−
;2. Does it have a block declaration/control?
+
|<code>usrp_x300_fpga_HG.bit</code>
−
: Well, it has the files now, so write your functions if you need them. Don't even worry about modifying the files to add your NoC-ID: the tool will do that for you with the ID of your choice.  
+
<code>usrp_x300_fpga_XG.bit</code>
  
−
;3 Does it have GNURadio integration?
+
<code>usrp_x310_fpga_HG.bit</code>
−
:It has it now. You just need to put additional control, in case you need it.
+
  
 +
<code>usrp_x310_fpga_XG.bit</code>
 +
|<code>2x DDC, 2x DUC</code>
 +
|-
  
−
Don't worry about relocating your files, they are where they need to be: '''in your custom location.''' The tool generates the files so that the whole building process take into account your design regardless of the location of your OOT module in your machine.
+
|<code>usrp_x300_fpga_RFNOC_HG.bit</code>
 +
<code>usrp_x300_fpga_RFNOC_XG.bit</code>
 +
|<code>DUC, DDC (one channel), fosphor, window, fft, 2x AXI FIFOs</code>
 +
|-
  
 +
|<code>usrp_x310_fpga_RFNOC_HG.bit</code>
 +
<code>usrp_x310_fpga_RFNOC_XG.bit</code>
 +
|<code>DUC, DDC (one channel), fosphor, window, fft, 2x AXI FIFOs, Keep One in N, FIR, Siggen</code>
 +
|-
  
 +
|<code>usrp_e310_fpga.bit</code>
 +
<code>usrp_e310_fpga_sg3.bit</code>
 +
|<code>1x DDC, 1x DUC</code>
 +
|-
  
−
=== Tool Installation ===
+
|<code>usrp_e310_fpga_RFNOC.bit (sg1 version)</code>
 +
<code>usrp_e310_fpga_RFNOC_sg3.bit</code>
 +
|<code>fosphor, window, fft, 2x AXI FIFOs, FIR</code>
 +
|-
  
−
The tool is currently shipped within gr-ettus (radio-redo branch), so by installing gr-ettus, rfnocmodtool automatically becomes available.
+
|}
  
−
=== Tool Utilization ===
+
Instructions on flashing the image to a device can be found [http://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_load_fpga_imgs here] for X3xx series and [http://files.ettus.com/manual/page_usrp_e3x0.html#e3x0_load_fpga_imgs here] for E3xx series.
 +
 
 +
'''NOTE:''' FPGA images are specific to the USRP device '''NOT''' the USRP series. For example, a USRP X300 FPGA image will '''NOT''' work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device.
  
−
After the installation, you are ready to use the tool to generate standard code for your Out-of-tree module. To check the usage of the tool, just type:
+
By following the steps above the following should now be available:
 +
* UHD/RFNoC code downloaded and installed
 +
* FPGA code available
 +
* A valid RFNoC image on your X3xx or E3xx series device
 +
 
 +
====Inspect default images====
 +
Run the following command, with a USRP connected to your PC, to verify current image on the USRP.
 +
 
 +
    $ uhd_usrp_probe
 +
 
 +
If an RFNoC image was successfully loaded onto the USRP, there will be a lot of output text (RFNoC code is currently very verbose). The final lines of the output should be similar to the following for an USRP X310 ( e.g. <code>usrp_x310_fpga_HG</code> ):
 +
 
 +
    |  |    _____________________________________________________
 +
    |  |    /
 +
    |  |  |      RFNoC blocks on this device:
 +
    |  |  | 
 +
    |  |  |  * DmaFIFO_0
 +
    |  |  |  * Radio_0
 +
    |  |  |  * Radio_1
 +
    |  |  |  * DDC_0
 +
    |  |  |  * DDC_1
 +
    |  |  |  * DUC_0
 +
    |  |  |  * DUC_1
 +
 
 +
 
 +
Final output for <code>usrp_x310_fpga_RFNOC_HG.bit</code> image:
 +
 
 +
    |  |    _____________________________________________________
 +
    |  |    /
 +
    |  |  |      RFNoC blocks on this device:
 +
    |  |  | 
 +
    |  |  |  * DmaFIFO_0
 +
    |  |  |  * Radio_0
 +
    |  |  |  * Radio_1
 +
    |  |  |  * DDC_0
 +
    |  |  |  * DUC_0
 +
    |  |  |  * FFT_0
 +
    |  |  |  * Window_0
 +
    |  |  |  * FIR_0
 +
    |  |  |  * SigGen_0
 +
    |  |  |  * KeepOneInN_0
 +
    |  |  |  * fosphor_0
 +
    |  |  |  * FIFO_0
 +
    |  |  |  * FIFO_1
 +
 
 +
 
 +
'''NOTE:''' The actual names and number of blocks can differ. The list of blocks should start with the <code>DmaFIFO_x</code> and <code>Radio_x</code>, and then a couple more lines of block IDs should follow.
 +
 
 +
====Build custom image with pre-built RFNoC blocks====
 +
Because of the growing number of RFNoC blocks, the user has the option to build an FPGA image with a set of pre-built RFNoC blocks of their choosing. The following steps describe the process for doing this and by so doing will also validate proper tool installation. Because compilation can take a couple of hours, it is recommended the user begin this process while continuing the rest of this guide.
 +
 
 +
'''NOTE:''' FPGA compilations can run in the background, however they are very resource intensive. If the user intents to use the same computer that is compiling to walk through the rest of this Application Note, it is recommended that the computer has plenty of resources.
 +
 
 +
The script to initiate a compile is called <code>uhd_image_builder.py</code>, and is located in the <code>{USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts</code> directory. Run the help menu by typing:
 +
 
 +
    $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts
 +
    $ ./uhd_image_builder.py --help
 +
 
 +
A more detailed discussion of this script is given in an upcoming section. For now, compiling an FPGA image that has 2 RFNoC blocks (<code>window</code> and <code>fft</code>) and some <code>FIFOs</code>, is done by running the script with the following arguments.
 +
 
 +
Example for an X310 USRP:
 +
 
 +
    $ ./uhd_image_builder.py window fft -d x310 -t X310_RFNOC_HG -m 5 --fill-with-fifos
 +
 
 +
Example for an E310 USRP with Speed Grade 3 (sg3) FPGA:
 +
 
 +
    $ ./uhd_image_builder.py window fft -d e310 -t E310_RFNOC_sg3 -m 5 --fill-with-fifos
 +
 
 +
At the end of a successful compilation process, write the new image to a USRP. If the image was compiled for a USRP X310, the following command will load the new image. Update the <code>{IP_Address}</code> of the USRP and <code>{USER_PREFIX}</code> to the appropriate values for your configuration before running the command.
 +
 
 +
    $ uhd_image_loader --args "type=x300,addr={IP_ADDRESS}" --fpga-path {USER_PREFIX}/src/uhd-fpga/usrp3/top/x300/build/usrp_x310_fpga_RFNOC_HG.bit
 +
 
 +
'''NOTE:''' FPGA images are specific to the USRP device '''NOT''' the USRP series. For example, a USRP X300 FPGA image will '''NOT''' work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device. Additional instructions on flashing a custom image to a device can be found [http://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_load_fpga_imgs here] for X3xx series and [http://files.ettus.com/manual/page_usrp_e3x0.html#e3x0_load_fpga_imgs here] for E3xx series.
 +
 
 +
After the image has been successfully written to the USRP, power-cycle it and run the “<code>uhd_usrp_probe</code>” utility to view the newly compiled blocks.
 +
 
 +
    $ uhd_usrp_probe
 +
 
 +
The final lines of output for the image built for the X310 is as follows:
 +
 
 +
    |  |    _____________________________________________________
 +
    |  |    /
 +
    |  |  |      RFNoC blocks on this device:
 +
    |  |  | 
 +
    |  |  |  * DmaFIFO_0
 +
    |  |  |  * Radio_0
 +
    |  |  |  * Radio_1
 +
    |  |  |  * Window_0
 +
    |  |  |  * FFT_0
 +
    |  |  |  * FIFO_0
 +
    |  |  |  * FIFO_1
 +
    |  |  |  * FIFO_2
 +
 
 +
===Getting started with UHD + RFNoC===
 +
The following new examples included within the <code>rfnoc-devel</code> branch of UHD, are a good reference on how to use RFNoC from UHD.
 +
 
 +
The following example is based off of <code>rx_samples_to_file.cpp</code>. The example can be configured to place an RFNoC block in between the radio and host.
 +
 
 +
    rfnoc_rx_to_file.cpp
 +
 
 +
This next example chains a null source to another block and streams the data to the host.
 +
 
 +
    rfnoc_nullsource_ce_rx.cpp
 +
 
 +
These examples demonstrate the core features and flexibility of RFNoC.
 +
 
 +
For more information on UHD and UHD development please refer to the [https://kb.ettus.com/UHD UHD Software Resource page], [https://kb.ettus.com/Getting_Started_with_UHD_and_C%2B%2B Getting Started with UHD and C++ Application Note] or directly to the [http://files.ettus.com/manual/ UHD user manual].
 +
 
 +
===Getting started with GNU Radio + RFNoC===
 +
A good way of getting started with RFNoC in a more visual way is to use GNU Radio. The <code>gr-ettus</code> out-of-tree module (OOT) allows a user to use RFNoC blocks in their local GNU Radio / GNU Radio Companion (GRC) installation. This GNU Radio OOT contains blocks that allow you to configure your FPGA through GRC.
 +
 
 +
'''NOTE:''' As blocks in the <code>gr-ettus</code> OOT mature, they will be upstreamed to <code>gr-uhd</code>. Also, <code>gr-ettus</code> is a container used by Ettus Research to disseminate experimental or under-development features for <code>gr-uhd</code>. It is not a replacement for <code>gr-uhd</code> (in fact, the latter is a requirement for <code>gr-ettus</code>).
 +
   
 +
Examples can be run from <code>gr-ettus/examples/rfnoc</code>, provided that the appropriate RFNoC blocks are compiled into the FPGA image currently running on the USRP.
 +
 
 +
A couple of rules for building GNU Radio flowgraphs with RFNoC blocks:
 +
 
 +
* You always need a <code>Device3</code> object in your flow graph (it does not get connected, see screenshot below).
 +
* You should have at least two RFNoC blocks connected together. Going <code>GNU Radio Block</code> -> <code>RFNoC Block</code> -> <code>GNU Radio Block</code> is not recommended (it will work, but with suboptimal performance).
 +
 
 +
The GNU Radio flowgraph <code>rfnoc_ddc.grc</code> is an example that can be run using the default RFNoC image. Below are screenshots of the flowgraph and what it produces.
 +
 
 +
[[File:rfnoc gsg an 1.png|center|800px|]]
 +
 
 +
'''NOTE:''' Copy Block: In the RFNoC domain, streams of data can not be split as easily as they are in the GNU Radio domain. The “copy” block depicted in the screenshot above serves the function as a stream splitter. Its main purpose, when “enabled”, is to copy the samples it is getting at its input and put them into the output, but here it is also serving as a boundary between a RFNoC-domain and a GNURadio-domain. In the flowgraph above, after this boundary is passed, the data stream can easily be split into the two sinks to have them run simultaneously (standard GNU Radio functionality). It is possible to connect the GNU Radio blocks directly to RFNoC blocks without a “copy” block, but only one would work at a time (the other ones would have to be disabled). Another way to split data streams from the RFNoC-domain is to use the ”RFNoC: split stream” block, which would split the streams in the RFNoC domain, but this is not very useful here as we are, in any case, moving into the GNURadio-domain.
 +
 
 +
[[File:rfnoc gsg an 2.png|center|800px|]]
 +
 
 +
For more information on GNURadio development please refer to the [http://gnuradio.org/doc/doxygen/ GNURadio user's manual and API].
 +
 
 +
==Starting a custom RFNoC block using RFNoC Modtool==
 +
The figure below shows the basic structure of the RFNoC Stack. Corresponding code is needed in each of the three sections in order to build a custom RFNoC block with GNU Radio integration. A tool called RFNoC Modtool was created in order to minimize the effort needed to implement a new RFNoC block. RFNoC Modtool creates a custom GNU Radio OOT module with the basic structure and the necessary files for each of these sections. RFNoC Modtool is currently a part of the GNU Radio OOT module <code>gr-ettus</code>.
 +
 
 +
[[File:rfnoc gsg an 3.png|center|800px|]]
 +
 
 +
===RFNoC Modtool Utilization===
 +
'''NOTE:''' Console outputs may vary depending on the version of UHD the user is running. However, functionality should be the same or similar.
 +
 
 +
Because the RFNoC Modtool has similar functionality to the <code>gr_modtool</code> [ [http://gnuradio.org/redmine/projects/gnuradio/wiki/OutOfTreeModules gr_modtool] ] provided by GNU Radio, those that have worked with <code>gr_modtool</code> in the past will find the RFNoC Modtool familiar.
 +
 
 +
To check the usage of the tool, run the following:
  
 
     $ rfnocmodtool help
 
     $ rfnocmodtool help
 +
    linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
 
      
 
      
−
    linux; GNU C++ version 4.8.4; Boost_105400; UHD_003.010.rfnoc-radio-redo-0-fd4d734e
 
 
     Usage:
 
     Usage:
 
     rfnocmodtool <command> [options] -- Run <command> with the given options.
 
     rfnocmodtool <command> [options] -- Run <command> with the given options.
Line 126: Line 371:
 
     add      insert          Add block to the out-of-tree module.  
 
     add      insert          Add block to the out-of-tree module.  
 
     newmod    nm,create        Create a new out-of-tree module  
 
     newmod    nm,create        Create a new out-of-tree module  
−
     rename    mv              Rename a block in the out-of-tree module.  
+
     rename    mv              Rename a block in the out-of-tree module.
  
−
If you have used the gr_modtool provided by GNURadio, you'll be familiar with this usage.
+
===Creating an RFNoC OOT Module===
−
 
+
−
== Creating an RFNoC OOT Module ==
+
−
 
+
−
At this point you are all set up to start generating your own RFNoC OOT and if you were thinking that it is just as easy as it is with gr_modtool, you are completely right! Simply go to your prefixes source location (~/prefix/rfnoc/src) and type:
+
  
 +
To start generating an RFNoC OOT module navigate to the source location ( i.e. <code>cd ~/{USER_PREFIX}/src</code> ) and type:
 
     $ rfnocmodtool newmod [NAME OF THE MODULE]
 
     $ rfnocmodtool newmod [NAME OF THE MODULE]
  
−
In the following, assume we want to create a module with the name 'example'. You can write the name of your module right after the 'newmod' command, but if you happen to forget it, the tool will ask for it interactively. This will create a folder containing the basic folders that you may need for a functional module.
 
  
−
     $ rfnocmodtool newmod example
+
Where <code>[NAME OF THE MODULE]</code> is a name the user gives the new module. In the following, a module is created with the name “<code>tutorial</code>”. If the user does not write the name of the module following the <code>newmod</code> command the tool will ask for it interactively. Running this command will create a folder containing the basic folders that you may need for a functional module.
−
     linux; GNU C++ version 4.8.4; Boost_105400; UHD_003.010.rfnoc-radio-redo-0-fd4d734e
+
 
 +
     $ rfnocmodtool newmod tutorial
 +
     linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
 
      
 
      
−
     Creating out-of-tree module in ./rfnoc-example... Done.
+
     Creating out-of-tree module in ./rfnoc-tutorial... Done.
 
     Use 'rfnocmodtool add' to add a new block to this currently empty module.
 
     Use 'rfnocmodtool add' to add a new block to this currently empty module.
−
   
 
−
    $ ls rfnoc-example/
 
−
    $ apps  cmake  CMakeLists.txt  docs  examples  grc  include  lib  MANIFEST.md  python  README.md  rfnoc  swig
 
  
−
In contrast with gr_modtool, this includes a folder called 'rfnoc', which is where we put the UHD/FPGA files (more details below).
+
To see what files and directories were created run:
  
−
=== Adding custom blocks ===
+
    $ ls rfnoc-tutorial/
 +
    apps  cmake  CMakeLists.txt  docs  examples  grc  include  lib  MANIFEST.md  python  README.md  rfnoc  swig
  
−
Now you can start adding blocks to your module. The only thing to do is go inside of the module you just created and type:
 
  
−
    $ rfnocmodule add [NAME OF THE BLOCK]
+
In contrast with <code>gr_modtool</code>, this includes a folder called <code>rfnoc</code>, which is where the UHD/FPGA files are located.
  
−
For demonstrative purposes, lets create a block named 'copy'. As for the new module, the tool will ask you for the name of the block if you not put it in the first line. There is a bunch of directives that you could pass to the tool, but if you run it without any of these directives you should expect the following interactive parsing output:
+
===Adding custom blocks to OOT Module===
 +
In order to add blocks to a module, navigate to the folder just created and use the <code>add</code> command of <code>rfnocmodtool</code>. Continuing with the example above, run the following:
  
−
     $ rfnocmodtool add copy
+
    $ cd rfnoc-tutorial
−
     linux; GNU C++ version 4.8.4; Boost_105400; UHD_003.010.rfnoc-radio-redo-0-fd4d734e
+
     $ rfnocmodtool add [NAME OF THE BLOCK]
 +
 
 +
For demonstrative purposes, a block named <code>gain</code> will be created. The <code>gain</code> block will multiply samples that pass through it by a constant. As before, if the name is not given, the tool will ask the user for the name. There are several arguments that can be passed to the tool, but running the tool without any of these arguments will give the following interactive parsing output:
 +
 
 +
    $ rfnocmodtool add gain
 +
     linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
 
      
 
      
−
     RFNoC module name identified: example
+
     RFNoC module name identified: tutorial
−
     Block/code identifier: copy
+
     Block/code identifier: gain
 
     Enter valid argument list, including default arguments:  
 
     Enter valid argument list, including default arguments:  
−
    Add Python QA code? [Y/n]
+
     Block NoC ID (Hexadecimal): 1111222233334444
−
    Add C++ QA code? [y/N]
+
     Skip Block Controllers Generation? [UHD block ctrl files] [y/N] N
−
     Block NoC ID (Hexadecimal): FFFF
+
     Skip Block interface files Generation? [GRC block ctrl files] [y/N] N
−
     Skip Block Controllers Generation? [y/N]  
+
−
     Skip Block interface Generation? [y/N]  
+
  
−
The tool is going to ask you about a valid argument list, in case you have that clear at this stage of your design, and for the generation of QA codes which are, such as in the case of gr_modtool, optional. Right after that, specific RFNoC questions appear:
+
Hitting <code>enter</code> on each one of the options will take the default values.
  
−
* NoC ID: This ID is a Hexadecimal number which serves as identification between the hardware part and the software part of the design. It can be as long as 16 0-9 A-F digits. If you dont provide the NoC ID, it will be set to a random number.
+
The following is a description of the valid argument list items:
−
* Block Controllers Generation: The block controllers are the C++ control that you can apply to the UHD-part of the design. In this files, you can add more control over this layer of your design. However, if you can manage to add the control you need over NoC-Script, you can omit the cpp/hpp block control files generation, reason why this is optional.
+
−
* Block Interface: Analogously to the block controllers, you may want to add more design specific functionality to your design at the GNURadio interface. To do that, you add the logic into the block-interface files. In the same manner, if your control can be managed by adding NoC-Script to your xml file, you can disregard the generation of files.
+
  
−
Also, it is important to notice that if you don't intend to use the block controllers or you are not sure if you would, the presence
+
* '''NoC ID:''' This ID is a Hexadecimal number which serves as identification between the hardware part and the software part of the design. It can be as long as 16 0-9 A-F digits. If a NoC ID is not provided, it will be set to a random number.
  
−
     Adding file 'lib/copy_impl.h'...
+
* '''Block Controllers Generation:''' The block controllers are the C++ control that the user can apply to the UHD-part of the design. In these files, the user can add more control over this layer of the design. Depending on the complexity of the block it may be possible to add all necessary control using NoCScript (more details on NoCScript can be found in the section labeled UHD Integration). In this case the cpp/hpp block control files generation are not needed. Default is to generate as their existence will be ignored if not edited.
−
     Adding file 'lib/copy_impl.cc'...
+
 
−
     Adding file 'include/example/copy.h'...
+
* '''Block Interface:''' Add more design specific functionality to the design at the GNU Radio interface by generating these block-interface files and adding necessary logic.  Depending on the complexity of the block it may be possible to add all necessary control using NoC-Script. In this case the block-interface files are not needed. Default is to generate as their existence will be ignored if not edited.
−
     Adding file 'include/example/copy_block_ctrl.hpp'...
+
 
−
     Adding file 'lib/copy_block_ctrl_impl.cpp'...
+
'''NOTE:''' If the user does not intend to use the block controllers or is not sure if they are needed, the presence of them in the design will do no harm. It is recommended to add them. This leaves the possibility to add more functions inside them in a future stage of development.
−
     Editing swig/example_swig.i...
+
 
−
     Adding file 'python/qa_copy.py'...
+
After finishing the parsing, the following files will be generated/edited:
 +
 
 +
     Adding file 'lib/gain_impl.h'...
 +
     Adding file 'lib/gain_impl.cc'...
 +
     Adding file 'include/tutorial/gain.h'...
 +
     Adding file 'include/tutorial/gain_block_ctrl.hpp'...
 +
     Adding file 'lib/gain_block_ctrl_impl.cpp'...
 +
     Editing swig/tutorial_swig.i...
 +
     Adding file 'python/qa_gain.py'...
 
     Editing python/CMakeLists.txt...
 
     Editing python/CMakeLists.txt...
−
     Adding file 'grc/example_copy.xml'...
+
     Adding file 'grc/tutorial_gain.xml'...
−
     Adding file 'rfnoc/blocks/copy.xml'...
+
     Adding file 'rfnoc/blocks/gain.xml'...
−
     Adding file 'rfnoc/fpga-src/noc_block_copy.v'...
+
     Adding file 'rfnoc/fpga-src/noc_block_gain.v'...
 +
    rfnoc/testbenches/noc_block_gain_tb folder created
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv'...
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/Makefile'...
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/CMakeLists.txt'...
 +
 
 +
==Creating FPGA portion of custom RFNoC Block==
 +
===RFNoC FPGA User Interface (API)===
 +
RFNoC blocks or Computation Engines (CEs) in the FPGA use a NoC Shell instance to interface with the rest of RFNoC. NoC Shell implements RFNoC's core functionality: packet muxing and demuxing, flow control, and the settings register bus (i.e. write/read control/status registers). The NoC Shell has an interface to the RFNoC AXI stream crossbar and a user interface. NoC Shell AXI stream interfaces expect CHDR packets with a proper header. See the manual for information on [https://files.ettus.com/manual/page_rtp.html CHDR and SID].
 +
 
 +
'''NOTE:''' AXI Stream is an ARM AMBA standard interface. Xilinx has an [http://www.xilinx.com/support/documentation/ip_documentation/ug761_axi_reference_guide.pdf AXI Reference Guide] with more details on this standard.
 +
 
 +
[[File:rfnoc gsg an 4.png|center|800px|]]
 +
 
 +
Many designs will want to use an AXI Stream interface with only sample data. However, as stated earlier, the NoC Shell block expects CHDR packets. To ease interfacing user code, the AXI Wrapper block provides the necessary logic to strip and insert the CHDR header, effectively converting packetized sample data into streaming sample data and vice versa. The example RFNoC blocks <code>noc_block_fft.v</code> and <code>noc_block_fir.v</code> show how AXI Wrapper is used to implement existing Xilinx AXI Stream based IP within a computation engine.
 +
 
 +
'''NOTE:''' AXI Wrapper also supports AXI Stream buses for configuration. These buses are driven via the setting register bus and do not have back pressure. They also consume two user register addresses per bus.
 +
 
 +
The primary user interface consists of four AXI stream interfaces ( <code>tready, tvalid, tlast, tdata</code> ) and a settings register bus ( 8-bit, valid user register addresses: <code>128-255</code> ).
 +
 
 +
 
 +
{|
 +
|
 +
AXI Stream signals:
 +
* '''m_axis_data_tdata:''' Input sample data packets
 +
** Data coming from host or another CE
 +
* '''s_axis_data_tdata:''' Output sample data packets
 +
** Data going to another CE or host
 +
* '''m_axis_data_tready:''' Input signal to CE
 +
** Used to notify CE that downstream CE is ready for data
 +
* '''s_axis_data_tready:''' Output signal to CE
 +
** Used to notify upstream CE that CE is ready for data
 +
* '''m_axis_data_tvalid:''' Input signal to CE
 +
** Used to indicate upstream CE has valid data
 +
* '''s_axis_data_tvalid:''' Output signal to CE
 +
** Used to indicate to downstream CE that CE has valid data
 +
* '''m_axis_data_tlast:''' Input signal to CE
 +
** Used to delimit packets from upstream CE
 +
* '''s_axis_data_tlast:''' Output signal to CE
 +
** Used to delimit packets to downstream CE
 +
| style="vertical-align:top" |[[File:rfnoc gsg an 5.png|right|500px|]]
 +
|-
 +
|}
 +
 
 +
[[File:rfnoc gsg an 6.png|center|800px|]]
 +
 
 +
 
 +
{|
 +
|
 +
Settings Bus signals:
 +
* '''set_stb:''' Assert to write '''set_data''' to register at '''set_addr'''ess
 +
* '''set_addr:''' Register address to set
 +
* '''set_data:''' Data to set
 +
* '''rb_data:''' Data to read back
 +
* '''rb_strobe:''' Assert to read '''rb_data''' from register at '''set_addr'''ess
 +
 
 +
| style="vertical-align:top" |[[File:rfnoc gsg an 7.png|right|500px|]]
 +
|-
 +
|}
 +
For the <code>gain</code> example block the following architecture is desired:
 +
 
 +
[[File:rfnoc gsg an 8.png|center|800px|]]
 +
 
 +
Open the file <code>rfnoc-tutorial/rfnoc/fpga-src/noc_block_gain.v</code> that contains the RFNoC block skeleton code that was created when the <code>$ rfnocmodtool add gain</code> command was run and modify the following ('''BOLD''' indicates changes to the skeleton code).
 +
 
 +
    '''localparam [7:0] SR_GAIN = SR_USER_REG_BASE;'''
 +
    localparam [7:0] SR_TEST_REG_1 = SR_USER_REG_BASE + 8'd1;
 +
   
 +
    '''wire [15:0] gain;'''
 +
    '''setting_reg #('''
 +
      '''.my_addr(SR_GAIN), .awidth(8), .width(16))'''
 +
    '''sr_gain ('''
 +
      '''.clk(ce_clk), .rst(ce_rst),'''
 +
      '''.strobe(set_stb), .addr(set_addr), .in(set_data), .out(gain), .changed());'''
 +
   
 +
    always @(posedge ce_clk) begin
 +
        case(rb_addr)
 +
          '''8'd0 : rb_data <= {48'd0, gain};'''
 +
          8'd1 : rb_data <= {32'd0, test_reg_1};
 +
          default : rb_data <= 64'h0BADC0DE0BADC0DE;
 +
        endcase
 +
    end
 +
   
 +
    '''wire [31:0] pipe_in_tdata;'''
 +
    '''wire pipe_in_tvalid, pipe_in_tlast;'''
 +
    '''wire pipe_in_tready;'''
 +
 +
    '''wire [31:0] pipe_out_tdata;'''
 +
    '''wire pipe_out_tvalid, pipe_out_tlast;'''
 +
    '''wire pipe_out_tready;'''
 +
 +
    '''// Adding FIFO to ensure Pipeline'''
 +
    '''axi_fifo_flop #(.WIDTH(32+1))'''
 +
    '''pipeline0_axi_fifo_flop ('''
 +
      '''.clk(ce_clk),'''
 +
      '''.reset(ce_rst),'''
 +
      '''.clear(clear_tx_seqnum),'''
 +
      '''.i_tdata({m_axis_data_tlast,m_axis_data_tdata}),'''
 +
      '''.i_tvalid(m_axis_data_tvalid),'''
 +
      '''.i_tready(m_axis_data_tready),'''
 +
      '''.o_tdata({pipe_in_tlast,pipe_in_tdata}),'''
 +
      '''.o_tvalid(pipe_in_tvalid),'''
 +
      '''.o_tready(pipe_in_tready));''' 
 +
 +
    '''wire [15:0] i = pipe_in_tdata[31:16];'''
 +
    '''wire [15:0] q = pipe_in_tdata[15:0];'''
 +
 +
    '''wire [31:0] i_mult_gain = i*gain;'''
 +
    '''wire [31:0] q_mult_gain = q*gain;'''
 +
 +
    '''wire [31:0] mult_gain = {i_mult_gain[15:0], q_mult_gain[15:0]};'''
 +
    '''axi_fifo_flop #(.WIDTH(32+1))'''
 +
    '''pipeline1_axi_fifo_flop ('''
 +
      '''.clk(ce_clk),'''
 +
      '''.reset(ce_rst),'''
 +
      '''.clear(clear_tx_seqnum),'''
 +
      '''.i_tdata({pipe_in_tlast,mult_gain}),'''
 +
      '''.i_tvalid(pipe_in_tvalid),'''
 +
      '''.i_tready(pipe_in_tready),'''
 +
      '''.o_tdata({pipe_out_tlast,pipe_out_tdata}),'''
 +
      '''.o_tvalid(pipe_out_tvalid),'''
 +
      '''.o_tready(pipe_out_tready));'''
 +
 +
    '''/* Output Signals */'''
 +
    '''assign pipe_out_tready = s_axis_data_tready;'''
 +
    '''assign s_axis_data_tvalid = pipe_out_tvalid;'''
 +
    '''assign s_axis_data_tlast  = pipe_out_tlast;'''
 +
    '''assign s_axis_data_tdata  = pipe_out_tdata;'''
 +
 
 +
The following is a block diagram of the code created by the above Verilog:
 +
 
 +
[[File:gain_block_diagram_v01.png|center|800px|]]
 +
 
 +
'''NOTE:'''  In order to meet timing, FIFO blocks were added to either side of the Multiplication process.
 +
 
 +
===Creating and running HDL testbenches===
 +
In order to make the coding iteration process more efficient, it is recommended to create testbenches for all RFNoC blocks before compiling them into the FPGA image. This allows for flaw and/or bug detection early in the design. RFNoC Modtool provides the structure and files ( e.g. noc_block_{USER_BLOCK_NAME}_tb ) for the testbenches of each of the OOT blocks that are added with the <code>$ rfnocmodtool add</code> command.
 +
 
 +
Below is a figure that shows the general testbench architecture  that is created by the RFNoC Modtool. This architecture allows a user to test their custom block in the exact same environment it will be placed in when it is built into the RFNoC architecture. Other benefits of the testbench architecture include:
 +
* Testing through multiple blocks (e.g. FILTER -> FFT -> AVE)
 +
* Testing with multiple streams (e.g. RFNoC block ADD/SUB takes 2 streams, one that will have a constant added to it and one that will have a constant subtracted from it)
 +
* Data transfer abstraction (e.g. RFNoC Sim Lib API calls to <code>tb_streamer.send</code> and  <code>tb_streamer.recv</code> which take care of all the AXI stream signaling)
 +
 
 +
[[File:rfnoc gsg an 9.png|center|800px|]]
 +
 
 +
'''NOTE:''' The <code>noc_block_tb</code> block is an instantiation of the <code>noc_block_export_io</code> that is used in testbenches to communicate to the RFNoC architecture. This makes it possible to talk “RFNoC” to the user’s custom block and as such the custom block has a complete RFNoC experience (signaling, flowcontrol, addressing, etc)
 +
 
 +
From the [[Getting Started with RFNoC Development#Adding_custom_blocks_to_OOT_Module|Adding custom blocks to OOT Module section]] where the <code>gain</code> block was initially created, the last files generated were:
 +
 
 +
    rfnoc/testbenches/noc_block_gain_tb folder created
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv'...
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/Makefile'...
 +
    Adding file 'rfnoc/testbenches/noc_block_gain_tb/CMakeLists.txt'...
 +
 
 +
 
 +
The <code>noc_block_gain_tb</code> is a folder generated to contain all the files related to the test bench of the <code>gain</code> block. Each time a new OOT block is created, a new folder will be generated as well.
 +
 
 +
Inside of this folder are the following three files:
 +
 
 +
* <code>CMakeLists.txt:</code> this is an empty file used, so far, only to increase the scope of the compilers.
 +
* <code>noc_block_gain_tb.sv:</code> this is a ''System Verilog'' file, in which user custom tests are to be located.  This is the '''only''' file that needs to be modified.
 +
* <code>Makefile:</code> This file determines the directives that run the simulation.
 +
 
 +
 
 +
The <code>noc_block_gain_tb.sv</code> testbench skeleton code creates the following architecture:
 +
 
 +
[[File:testbench_arch_gain_v01.png|center|800px|]]
 +
 
 +
Open the file <code>rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv</code> and modify the following lines:
 +
 
 +
Right under the “Verification” section:
 +
 
 +
    initial begin : tb_main
 +
      string s;
 +
      logic [31:0] random_word;
 +
      logic [63:0] readback;
 +
      '''logic [15:0] gain;'''
 +
 
 +
In the “Test 4 -- Write / readback user registers” section:
 +
   
 +
    `TEST_CASE_START("Write / readback user registers");
 +
    random_word = $random();
 +
    '''tb_streamer.write_user_reg(sid_noc_block_gain, noc_block_gain.SR_GAIN, random_word[15:0]);'''
 +
    '''tb_streamer.read_user_reg(sid_noc_block_gain, 0, readback);'''
 +
    '''$sformat(s, "User register 0 incorrect readback! Expected: %0d, Actual %0d", readback[15:0], random_word[15:0]);'''
 +
    '''`ASSERT_ERROR(readback[15:0] == random_word[15:0], s);'''
 +
   
 +
In the “Test 5 -- Test sequence” section:
 +
 
 +
    `TEST_CASE_START("Test sequence");
 +
    '''gain = 100;'''
 +
    '''tb_streamer.write_user_reg(sid_noc_block_gain, noc_block_gain.SR_GAIN, gain);'''
 +
    fork
 +
      begin
 +
        cvita_payload_t send_payload;
 +
        for (int i = 0; i < SPP/2; i++) begin
 +
          send_payload.push_back(64'(i));
 +
        end
 +
        tb_streamer.send(send_payload);
 +
      end
 +
      begin
 +
        cvita_payload_t recv_payload;
 +
        cvita_metadata_t md;
 +
        logic [63:0] expected_value;
 +
        tb_streamer.recv(recv_payload,md);
 +
        for (int i = 0; i < SPP/2; i++) begin
 +
          '''expected_value = i*gain;'''
 +
 
 +
Test #4 verifies that we can write and readback the <code>gain</code> value. Test #5 writes to the <code>gain</code> register, sends a sample set in the form of a ramp (1, 2, 3, 4, etc) to the RFNoC gain block and finally reads the values from the <code>gain</code> block and compares them to expected values. The followings steps will allow the user to run this testbench.
 +
 
 +
From within the <code>rfnoc-tutorial</code> directory, create a <code>build</code> directory and enter it by running:
 +
 
 +
    $ mkdir build && cd build/
 +
 
 +
The next step is to run <code>cmake</code>. If PyBOMBS was used to create the development sandbox, <code>cmake</code> will automatically detect the location of the <code>fpga</code> repository. If PyBOMBS was not used, the user must provide the location of where the <code>fpga</code> repository is installed.
 +
 
 +
If PyBOMBS used, run:
 +
 
 +
    $ cmake ../
 +
 
 +
If PyBOMBS not used, run:
 +
 
 +
    $ cmake [-DUHD_FPGA_DIR=/PATH/TO/FPGA/REPOSITORY] ../
 +
 
 +
Final output from the <code>$ cmake ../</code> command:
 +
 
 +
    -- Configuring done
 +
    -- Generating done
 +
    -- Build files have been written to: /home/widow/rfnoc/src/rfnoc-tutorial/build
 +
 
 +
The following command will modify the necessary files and set the correct path to the simulation tools. From now on, every time a new block is added, this command will be run automatically. Remember, only run the following command once for each OOT module (not RFNoC block, but OOT module) created:
 +
 
 +
    $ make test_tb
 +
    Scanning dependencies of target test_tb
 +
    Built target test_tb
 +
 
 +
Testbenches can be executed by running the command:
 +
 
 +
    $ make noc_block_[name_of_your_block]_tb
 +
 
 +
The gain block testbench can be run by running the following command:
 +
 
 +
    $ make noc_block_gain_tb
 +
 
 +
The simulation will start.  Final output should look like this:
 +
 
 +
 
 +
    ========================================================
 +
    TESTBENCH STARTED: noc_block_gain
 +
    ========================================================
 +
    [TEST CASE  1] (t=000000000) BEGIN: Wait for Reset...
 +
    [TEST CASE  1] (t=000001002) DONE... Passed
 +
    [TEST CASE  2] (t=000001002) BEGIN: Check NoC ID...
 +
    Read GAIN NOC ID: 1111222233334444
 +
    [TEST CASE  2] (t=000001238) DONE... Passed
 +
    [TEST CASE  3] (t=000001238) BEGIN: Connect RFNoC blocks...
 +
    Connecting noc_block_tb (SID: 1:0) to noc_block_gain (SID: 0:0)
 +
    Connecting noc_block_gain (SID: 0:0) to noc_block_tb (SID: 1:0)
 +
    [TEST CASE  3] (t=000005457) DONE... Passed
 +
    [TEST CASE  4] (t=000005457) BEGIN: Write / readback user registers...
 +
    [TEST CASE  4] (t=000006888) DONE... Passed
 +
    [TEST CASE  5] (t=000006888) BEGIN: Test sequence...
 +
    [TEST CASE  5] (t=000007633) DONE... Passed
 +
    ========================================================
 +
    '''TESTBENCH FINISHED: noc_block_gain'''
 +
    ''' - Time elapsed:  7700 ns'''           
 +
    ''' - Tests Expected: 5'''
 +
    ''' - Tests Run:      5'''
 +
    ''' - Tests Passed:  5'''
 +
    '''Result: PASSED''' 
 +
    ========================================================
 +
    $finish called at time : 7700 ns : File "/home/widow/rfnoc/src/rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv" Line 10
 +
    INFO: [USF-XSim-96] XSim completed. Design snapshot 'noc_block_gain_tb_behav' loaded.
 +
    INFO: [USF-XSim-97] XSim simulation ran for 1000000000us
 +
    launch_simulation: Time (s): cpu = 00:00:10 ; elapsed = 00:00:12 . Memory (MB): peak = 966.387 ; gain = 54.848 ; free physical = 3080 ; free virtual = 29888
 +
    # if [string equal $vivado_mode "batch"] {
 +
    #    puts "BUILDER: Closing project"
 +
    #    close_project
 +
    # } else {
 +
    #    puts "BUILDER: In GUI mode. Leaving project open."
 +
    # }
 +
    BUILDER: Closing project
 +
    ****** Webtalk v2015.4 (64-bit)
 +
      **** SW Build 1412921 on Wed Nov 18 09:44:32 MST 2015
 +
      **** IP Build 1412160 on Tue Nov 17 13:47:24 MST 2015
 +
        ** Copyright 1986-2015 Xilinx, Inc. All Rights Reserved.
 +
   
 +
    source /home/widow/rfnoc/src/rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/xsim_proj/xsim_proj.hw/webtalk/labtool_webtalk.tcl -notrace
 +
    INFO: [Common 17-206] Exiting Webtalk at Tue Jan 10 23:26:20 2017...
 +
    INFO: [Common 17-206] Exiting Vivado at Tue Jan 10 23:26:22 2017...
 +
    Built target noc_block_gain_tb
 +
 
 +
 
 +
 
 +
With every custom block created, a <code>make</code> directive will be available to run the simulation from the <code>build</code> directory.
 +
 
 +
===Building the FPGA image with a custom user block===
 +
In this section steps are given on how to initiate an FPGA build while incorporating the user’s custom RFNoC block. The first sections give general information on building RFNoC images. The remaining two sections show how to initiate FPGA builds using a command line interface and using a graphical interface (coming out soon), respectively.
 +
 
 +
====Discussion on number of blocks in an FPGA image====
 +
There is a maximum number of blocks that can be added for each device. The maximum amount of computation engines (CEs/RFNoC blocks) that each device can use is 16, but the amount of custom blocks that can be added depends on the device.
 +
 
 +
If using a device from the X3xx series, from the 16 CEs, there are 6 that will be always added and are not subject to direct customization: 1 CE for the AXI bus, 1 CE for the Ethernet Interface, 2 Radios and 2 Dma FIFOS. Because of this, the application will only allow a number of 10 custom blocks on the X3xx series.
 +
 
 +
If using a device from the E3xx series, 2 CE engines are always added and are not subject to direct customization: 1 CE for the AXI bus and 1 Radio. This would virtually allow 14 slots for custom blocks. However, given the size of the FPGA on the E3xx series of devices, the application only allows a number of 6 custom blocks.
 +
 
 +
'''NOTE:''' Blocks with higher resource utilization may fill up the FPGA and force the user to include less blocks.
 +
 
 +
Verify the current maximum values by running the <code>uhd_images_builder.py</code> utility from the scripts directory.
 +
 
 +
    $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts
 +
    $ uhd_image_builder.py --help
 +
 
 +
====Discussion on FPGA image targets====
 +
RFNoC target names follow the pattern <code>{DEVICE}_RFNOC_{BUILD_TYPE}</code> with the following build types:
 +
 
 +
* <code>HG:</code> 1GigE on SFP+ Port0, 10Gig on SFP+ Port
 +
* <code>XG:</code> 10GigE on both SFP+ ports
 +
* <code>HLS:</code> Vivado High Level Synthesis enabled
 +
* <code>sgX:</code> Speed grade for E300 devices (1 or 3)
 +
 
 +
Some examples are:
 +
* <code>X310_RFNOC_HG</code>
 +
* <code>X300_RFNOC_HG</code>
 +
* <code>X310_RFNOC_XG</code>
 +
* <code>X300_RFNOC_XG</code>
 +
* <code>X310_RFNOC_HLS_HG</code>
 +
* <code>X300_RFNOC_HLS_HG</code>
 +
* <code>E310_RFNOC</code> (this is for the speed grade 1 FPGA version of E310, append <code>_sg3</code> for speed grade 3)
 +
 
 +
'''NOTE:''' E310, E312 and E313 all have the same FPGA hardware and therefore will use the <code>E310_RFNOC_{BUILD_TYPE}</code> target. USRP E3xx devices have either <code>sg1</code> or <code>sg3</code> hardware, please visit [http://files.ettus.com/e3xx_images/README here] to find out how to differentiate.
 +
 
 +
Additional information about the build targets can be found at the build instructions for USRP3, found [http://files.ettus.com/manual/md_usrp3_build_instructions.html here].
 +
 
 +
====Image building using the command line====
 +
The script <code>uhd_image_builder.py</code> is used to generate the NoC block instantiation file and build the FPGA image. Run the help menu by typing:
 +
 
 +
    $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts
 +
    $ ./uhd_image_builder.py --help
 +
       
 +
    usage: uhd_image_builder.py [-h] [-I INCLUDE_DIR [INCLUDE_DIR ...]]
 +
                                [-m MAX_NUM_BLOCKS] [--fill-with-fifos]
 +
                                [-o OUTFILE] [-d DEVICE] [-t TARGET] [-g] [-c]
 +
                                [blocks [blocks ...]]
 +
   
 +
    Generate the NoC block instantiation file
 +
   
 +
    positional arguments:
 +
      blocks                List block names to instantiate.
 +
   
 +
    optional arguments:
 +
      -h, --help            show this help message and exit
 +
      -I INCLUDE_DIR [INCLUDE_DIR ...], --include-dir INCLUDE_DIR [INCLUDE_DIR ...]
 +
                            Path directory of the RFNoC Out-of-Tree module
 +
      -m MAX_NUM_BLOCKS, --max-num-blocks MAX_NUM_BLOCKS
 +
                            Maximum number of blocks (Max. Allowed for x310|x300:
 +
                            10, for e300: 6)
 +
      --fill-with-fifos    If the number of blocks provided was smaller than the
 +
                            max number, fill the rest with FIFOs
 +
      -o OUTFILE, --outfile OUTFILE
 +
                            Output /path/filename - By running this directive, you
 +
                            won't build your IP
 +
      -d DEVICE, --device DEVICE
 +
                            Device to be programmed [x300, x310, e310]
 +
      -t TARGET, --target TARGET
 +
                            Build target - image type [X3X0_RFNOC_HG,
 +
                            X3X0_RFNOC_XG, E310_RFNOC_sg3...]
 +
      -g, --GUI            Open Vivado GUI during the FPGA building process
 +
      -c, --clean-all      Cleans the IP before a new build
 +
 
 +
 
 +
Here are details on the usage of the script which is followed by an example:
 +
 
 +
'''Blocks:''' The first arguments are the names of RFNoC blocks that the user wants to have compiled into the new image which are separated by a space. They can be custom blocks from the user’s OOT module or from the ones that are provided from Ettus, or a combination. Blocks provided by Ettus Research are listed (among other sources necessary for the FPGA build) in the <code>{USER_PREFIX}/src/uhd-fpga/usrp3/lib/rfnoc/Makefile.srcs</code> file.
 +
 
 +
These blocks can be identified by the following pattern:
 +
 
 +
    noc_block_{NAME}.v
 +
 
 +
However, as all the RFNoC blocks have the same <code>noc_block_</code> prefix, for simplicity this prefix is omitted when listing the blocks in the <code>uhd_image_builder.py</code> utility. As an example of the incorrect and correct way of adding blocks, consider the following examples when adding the <code>noc_block_null_source_sink</code> and <code>noc_block_siggen</code> blocks:
 +
 
 +
Incorrect method: 
 +
 
 +
    $ ./uhd_image_builder.py noc_block_null_source_sink noc_block_siggen ...
 +
 
 +
Correct method:
 +
 
 +
    $ ./uhd_image_builder.py null_source_sink siggen ...
 +
 
 +
'''NOTE:''' Blocks generated by the RFNoC Modtool follow the same naming convention.
 +
 
 +
There is an increasing list of pre-built blocks. Here is a sample:
 +
 
 +
* <code>axi_fifo_loopback</code>
 +
* <code>axi_dma_fifo</code>
 +
* <code>fir_filter</code>
 +
* <code>fft</code>
 +
* <code>null_source_sink</code>
 +
* <code>schmidl_cox</code>
 +
* <code>packet_resizer</code>
 +
* <code>split_stream</code>
 +
* <code>vector_iir</code>
 +
* <code>addsub</code>
 +
* <code>window</code>
 +
* <code>keep_one_in_n</code>
 +
* <code>pfb</code>
 +
* <code>export_io</code>
 +
* <code>conv_encoder_qpsk</code>
 +
* <code>siggen</code>
 +
* <code>logpwr</code>
 +
* <code>fosphor</code>
 +
* <code>moving_avg</code>
 +
* <code>ddc</code>
 +
* <code>duc</code>
 +
 
 +
 
 +
RFNoC related blocks generally reside in <code>fpga/usrp3/lib/rfnoc/</code>.
 +
 
 +
{| class="wikitable" style="margin:auto;"
 +
 
 +
!Block
 +
!Filename
 +
!Description
 +
|-
 +
 
 +
|FIFO
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_axi_fifo_loopback.v noc_block_axi_fifo_loopback.v]
 +
|Simple FIFO loopback / passthrough block.
 +
|-
 +
 
 +
|FFT
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_fft.v noc_block_fft.v]
 +
|Xilinx coregen based Fast Fourier Transform up to length 4096.
 +
|-
 +
 
 +
|FIR
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_fir_filter.v noc_block_fir_filter.v]
 +
|Xilinx coregen based Finite Impulse Response Filter, 41 taps, reconfigurable tap coefficients.
 +
|-
 +
 
 +
 
 +
|Window
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_window.v noc_block_window.v]
 +
|Windowing block for use with FFT block.
 +
|-
 +
 
 +
|Vector IIR
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_vector_iir.v noc_block_vector_iir.v]
 +
|Single pole IIR with configurable coefficients that filters data along vectors (i.e. parallel streams of samples). Useful with FFT output.
 +
|-
 +
 
 +
|Keep One in N
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_keep_one_in_n.v noc_block_keep_one_in_n.v]
 +
|Keeps one packet every N packets.
 +
|-
 +
 
 +
|AddSub
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_addsub.v noc_block_addsub.v]
 +
|Example of using multiple block ports in a single RFNoC block to add and subtract streams.
 +
|-
 +
 
 +
|Null Source Sink
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_null_source_sink.v noc_block_null_source_sink.v]
 +
|Generates dummy packets and can consume packets at a configurable rate. Useful for testing.
 +
|-
 +
 
 +
|Packet Resizer
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_packet_resizer.v noc_block_packet_resizer.v]
 +
|Resizes input packets to a configurable size (larger or smaller than source packets).
 +
|-
 +
 
 +
|Split Stream
 +
|[https://github.com/EttusResearch/fpga/blob/maint/usrp3/lib/rfnoc/noc_block_split_stream.v noc_block_split_stream.v]
 +
|Replicates an input stream to a configurable number of output streams.
 +
|-
 +
 
 +
 
 +
|}
 +
 
 +
 
 +
'''NOTE:''' There is a restriction on the amount of blocks that can added into the FPGA image, see the section in this Application Note labeled [[Getting_Started_with_RFNoC_Development#Discussion_on_number_of_blocks_in_an_FPGA_image|Discussion on number of blocks in an FPGA image]] for more information.
 +
 
 +
<code>-I INCLUDE_DIR:</code> The <code>-I</code> directive provides the path to top OOT directory, which contains the users <code>rfnoc/fpga-src</code> directory which contains the custom blocks. This path is needed by the Xilinx Vivado tool. Inside the <code>fpga-src</code> directory there is a file called <code>Makefile.srcs</code> that contains the path of the OOT module and a list of all the custom OOT blocks. This is an auto generated file, which is amended every time a new block is added to the OOT module. Manually modifying this file is not recommended. If there are multiple OOT modules with various custom blocks that reside in different directories the way to include them all is by separating the different paths by a space (e.g. <code>-I /first/OOT/path/ /second/OOT/path/</code>).
 +
 
 +
'''IMPORTANT:''' Please be sure to terminate the path of your OOT with the "/" character. Otherwise the path might not be recognized.
 +
 
 +
<code>-d DEVICE:</code> The <code>-d</code> directive directs the script on which USRP device the build is for. If no <code>–d</code> is included the default is <code>–d x310</code>. Generation-3 USRPs and above all support RFNoC.
 +
 
 +
<code>-t TARGET:</code> The <code>–t</code> directive directs the script on which type of image to build for the chosen device. With each USRP device there are several build options to choose from. Detailed information about the build targets can be found at the build instructions for USRP3, found [http://files.ettus.com/manual/md_usrp3_build_instructions.html here]. If <code>-t</code> is not included, a default target will be chosen for the given device. For example, the default <code>X310_RFNOC_HG</code> target builds for the <code>–d x310</code> device. More details on targets can be found in the section of this Application Note labeled [[Getting Started with RFNoC Development#Discussion_on_FPGA_image_targets|Discussion on FPGA image targets]].
 +
 
 +
<code>-m MAX_NUM_BLOCKS:</code> The <code>–m</code> directive specifies the max number of RFNoC blocks to build on the FPGA image. An RFNoC image does not need to fill all available slots with RFNoC blocks.
 +
 
 +
<code>--fill-with-fifos:</code> The <code>--fill-with-fifos</code> directive will fill the empty RFNoC block slots with FIFOS. As an example, if a user indicates three RFNoC blocks by name and also specifies <code>–m 5</code> then the other two slots will be filed with FIFOs.
 +
 
 +
<code>-o OUTFILE:</code> With the <code>-o</code> directive, the RFNoC blocks instantiation file is generated and saved at the desired path with the given name for the user to inspect. The FPGA image will NOT build if this directive is provided. The purpose of the <code>uhd_image_builder.py</code> script is to auto generate an instantiation file and populate the source files needed for the Xilinx Vivado tool to build the FPGA image, however, it may be desirable to only see the effect of adding a custom OOT module in the <code>fpga/</code> directory, or for inspecting the instantiation file. When the directive is not provided the <code>rfnoc_ce_auto_inst_x3x0.v</code> file is overwritten and the FPGA image build process will start automatically (standard use).
 +
 
 +
<code>-g, --GUI:</code> Open Vivado GUI during the FPGA building process
 +
 
 +
<code>-c, --clean-all:</code> Cleans the IP before a new build
 +
 
 +
Here is how to create an X310 FPGA image incorporating the <code>gain</code> block that was created earlier in this Application Note:
 +
 
 +
    $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts   
 +
    $ ./uhd_image_builder.py gain ddc fft -I {USER_PREFIX}/src/rfnoc-tutorial/ -d x310 -t X310_RFNOC_HG -m 6 --fill-with-fifos
 +
 
 +
 
 +
At the end of a successful compilation process, write the new image to a USRP. The following command will load the new image. Update the <code>{IP_Address}</code> of the USRP and <code>{USER_PREFIX}</code> to the appropriate values for your configuration before running the command.
 +
 
 +
    $ uhd_image_loader --args "type=x300,addr={IP_ADDRESS}" --fpga-path {USER_PREFIX}/src/uhd-fpga/usrp3/top/x300/build/usrp_x310_fpga_RFNOC_HG.bit
 +
 
 +
'''NOTE:'''
 +
* The FPGA image building process may take over an hour.
 +
 
 +
* FPGA images are specific to the USRP device NOT the USRP series. For example, a USRP X300 FPGA image will '''NOT''' work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device. Additional instructions on flashing a custom image to a device can be found [http://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_load_fpga_imgs here] for X3xx series and [http://files.ettus.com/manual/page_usrp_e3x0.html#e3x0_load_fpga_imgs here] for E3xx series.
 +
 
 +
* [Environment setup] - The <code>uhd_image_builder.py</code> script will also set up the Xilinx Vivado environment by automatically running the <code>setupenv.sh</code> script located in the <code>{USER_PREFIX}/src/uhd-fpga/usrp3/top/{device}</code> directory. The <code>setupenv.sh</code> script assumes that Xilinx Vivado is installed in the default location of <code>/opt/Xilinx/Vivado</code>.  If the installation is in a different directory the <code>setupenv_base.sh</code> script will need to be modified. The script is located at: <code>{USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts/setupenv_base.sh</code>
 +
 
 +
Besides the custom <code>gain</code> block, a <code>DDC</code> and <code>FFT</code> block are also being added along with three <code>FIFOs</code>.  The <code>DDC</code>, <code>FFT</code> and <code>FIFO</code> blocks are already in the script's path and therefore do not need their path specified (they ship with the Ettus Research FPGA code). The reason three FIFOs are added is because the max number of blocks was specified to be 6 ( <code>-m 6</code> ) and since only 3 blocks were specifically named the other three slots are filled with FIFOs.
 +
 
 +
 
 +
[[File:rfnoc gsg an 10.png|center|800px|]]
 +
 
 +
'''NOTE:''' Instructions on flashing the image to a device can be found [http://files.ettus.com/manual/page_usrp_x3x0.html#x3x0_load_fpga_imgs here] for X3xx series and [http://files.ettus.com/manual/page_usrp_e3x0.html#e3x0_load_fpga_imgs here] for E3xx series. FPGA images are specific to the USRP device '''NOT''' the USRP series. For example, a USRP X300 FPGA image will '''NOT''' work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device.
 +
 
 +
Once the newly compiled image is loaded onto a USRP X3xx running the following command will show what RFNoC blocks are available on the FPGA:
 +
 
 +
    $ uhd_usrp_probe
 +
   
 +
   
 +
    |  |    _____________________________________________________
 +
    |  |    /
 +
    |  |  |      RFNoC blocks on this device:
 +
    |  |  | 
 +
    |  |  |  * DmaFIFO_0
 +
    |  |  |  * Radio_0
 +
    |  |  |  * Radio_1
 +
    |  |  |  * '''Block_0'''
 +
    |  |  |  * DDC_0
 +
    |  |  |  * FFT_0
 +
    |  |  |  * FIFO_0
 +
    |  |  |  * FIFO_1
 +
    |  |  |  * FIFO_2
 +
 
 +
 
 +
'''NOTE:''' The reason the custom block is called <code>Block_0</code> and not <code>gain_0</code> is because there is still host side software/files that need updated in order for this block to populate it’s proper name. A following section (UHD Integration) will step through the process of updating those host side files.
 +
 
 +
====Using a graphical interface====
 +
A graphical user interface for FPGA generation and building is shipped along with the <code>uhd_image_builder.py</code> script. This intuitive application aids in setting up a custom FPGA build.
 +
 
 +
This utility is located in the same <code>scripts</code> directory as <code>uhd_image_builder.py</code>.
 +
 
 +
To run it, enter the following commands:
 +
 
 +
    $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts/
 +
    $ ./uhd_image_builder_gui
 +
 
 +
The application will then be launched:
 +
 
 +
[[File:rfnoc gsg an 11.png|center|800px|]]
 +
 
 +
'''1. Select build target:''' In this panel the available build targets are listed. This list may vary depending on which branch of the FPGA repository this user is using. Only RFNoC targets are listed. The build type descriptions are:
 +
 
 +
* <code>HG:</code> 1GigE on SFP+ Port0, 10Gig on SFP+ Port1
 +
* <code>XG:</code> 10GigE on both SFP+ ports
 +
* <code>HLS:</code> Vivado High Level Synthesis enabled
 +
* <code>sgX:</code> Speed grade for E300 devices (1 or 3)
 +
 
 +
'''2. List of blocks available:''' In this panel the available blocks are listed that can be included into a custom design. This list separates the RFNoC blocks provided by Ettus Research and the OOT modules and corresponding blocks that the user adds. Given the hardware differences between the X3xx and E3xx devices, this list will dynamically change when a different device is selected from the panel on the left. This implies that it is necessary to add the OOT modules for each device independently. This is accomplished by using the <code>Add OOT Blocks</code> feature of the application, details of which are explained at #7 (<code>Add OOT Blocks</code>).
 +
 
 +
'''3. Blocks in current design:''' This section gives information on the MAX number of blocks for a given USRP (based on the target selection). There is a maximum number of blocks that can be added for each device. See the section in this App Note labeled "Discussion on number of blocks in an FPGA image" for more information.
 +
 
 +
'''4. Blocks in current design:''' This panel will be populated by adding elements from the available blocks. All the blocks listed in here will be compiled into the FPGA custom image. There is a maximum number of blocks that can be added for each device. See the section in this App Note labeled "Discussion on number of blocks in an FPGA image" for more information.
 +
 
 +
'''5. Add button (>>):''' Manually add the blocks from the central panel into your design.
 +
 
 +
'''6. Remove button (<<):''' Remove blocks from the current design (far-left panel)
 +
 
 +
'''7. Fill with FIFOs:''' By checking this box, the design will fill any available/unspecified block slots with FIFOs. The number of FIFO blocks that will be instantiated is based on the rules of amount of blocks explained at #3. When less than the max amount of blocks are needed for certain implementation, many users choose to fill their design with FIFO blocks.
 +
 
 +
'''8. Open Vivado GUI:''' Open Vivado GUI during the FPGA building process. This allows the user to save a Vivado project with all IP and work within the Vivado GUI for development.
 +
 
 +
'''9. Clean IP:''' Cleans the IP before a new build (recompiles all IP).
 +
 
 +
'''10. Add OOT blocks:''' Manually add RFNoC Modtool-generated OOT modules by pointing the application to the <code>Makefile.srcs</code> file, which is located in the <code>{USER_PREFIX}/src/{USER-OOT-moddir}/rfnoc/fpga-srcs/</code> directory. After adding this file, blocks will appear under “<code>OOT blocks for XXXX devices</code>”
 +
 
 +
'''11. Show Instantiation File:''' The application auto-generates the instantiation file that is going to be used by Vivado to build the FPGA image. This instantiation file can be viewed and edited before starting the build by clicking this button.
 +
 
 +
'''12. Import from GRC:''' If the user has a GNU Radio flowgraph with RFNoC blocks already in it, this application can read what RFNoC blocks are in the flowgraph and populate the <code>Blocks in current design</code> section of the application with the necessary RFNoC blocks. '''NOTE:''' All RFNoC blocks pulled from a <code>.grc</code> file must be in the of <code>List of blocks available</code> before beginning the build.
 +
 
 +
'''13. Generate .bit file:''' Start the build by clicking this button.
 +
 
 +
'''14. uhd_image_builder command:''' The command line command with arguments is dynamically build here as the user selects different options. The user could save this command to use next time they build/compile an FPGA image to avoid having to select all options again.
 +
 
 +
'''NOTE:''' See the latter end of the previous section for additional information on what to expect once the compile has started as well as final output.
 +
 
 +
==Creating Software/Host portion of custom RFNoC Block==
 +
Now that the FPGA portion is complete the next step is to add software integration to UHD and GNU Radio as depicted in the RFNoC Stack below.
 +
 
 +
[[File:rfnoc gsg an 12.png|center|800px|]]
 +
 
 +
===UHD integration===
 +
Despite the data processing happening on the FPGA, the host software still has a lot of responsibilities in order for an RFNoC application to function. For example, it needs to know which settings registers are available within an RFNoC block, or what kind of input and output a block has. All of this information goes into the <code>Block Declaration</code>, which is an XML file that is readable by UHD. Often, some simple logic needs to be embedded in the XML file, which we can do by using a simple scripting language called Noc-Script. Changes to the block declaration file are immediately imported into UHD every time an application is executed, and therefore, no software development toolchain needs to be set up.
 +
 
 +
The list of things declared by the block declaration file includes:
 +
 
 +
* Block name and Noc-ID
 +
* Registers
 +
* Inputs and outputs (including types)
 +
 
 +
In some cases, additional C++ code is required to properly control a block from software. In this case, a <code>Block Controller</code> file is required as well as the declaration file. In most cases, the default block controller provided by UHD is sufficient, so no C++ code needs to be written. Writing custom block controllers requires more effort, and means having to set up a programming toolchain. A common reason to write custom C++ block controllers is if setting a register requires a lot of computation, which is not feasible to do within a block declaration file (e.g., using Noc-Script).
 +
 
 +
Skeleton code for both the block declaration and the block controller (if required) can be generated through RFNoC Modtool.
 +
 
 +
Because the <code>gain</code> block does not require anything other than simply reading and writing to a single register the default block controller will suffice for this example. However, we will need to add information about the register.
 +
 
 +
Open the <code>gain.xml</code> file located in the <code>/rfnoc-tutorial/rfnoc/blocks</code> directory and add the following:
 +
 
 +
 
 +
    <?xml version="1.0"?>
 +
    <nowiki><!--Default XML file--></nowiki>
 +
    <nocblock>
 +
      <name>gain</name>
 +
      <blockname>gain</blockname>
 +
      <ids>
 +
        <id revision="0">1111222233334444</id>
 +
      </ids>
 +
      '''<nowiki><!-- Registers --></nowiki>'''
 +
      '''<registers>'''
 +
        '''<setreg>'''
 +
          '''<name>GAIN</name>'''
 +
          '''<address>128</address>'''
 +
        '''</setreg>'''
 +
      '''</registers>'''
 +
      '''<nowiki><!-- Args --></nowiki>'''
 +
      '''<args>'''
 +
        '''<arg>'''
 +
          '''<name>gain</name>'''
 +
          '''<type>double</type>'''
 +
          '''<value>1.0</value>'''
 +
          '''<check>GE($gain, 0.0) AND LE($gain, 32767.0)</check>'''
 +
          '''<check_message>Invalid gain.</check_message>'''
 +
          '''<action>'''
 +
            '''SR_WRITE("GAIN", IROUND($gain))'''
 +
          '''</action>'''
 +
        '''</arg>'''
 +
      '''</args>'''
 +
      <nowiki><!--One input, one output. If this is used, better have all the info the C++ file.--></nowiki>
 +
      <ports>
 +
        <sink>
 +
          '''<name>in0</name>'''
 +
          '''<type>sc16</type>'''
 +
        </sink>
 +
        <nowiki><source></nowiki>
 +
          '''<name>out0</name>'''
 +
          '''<type>sc16</type>'''
 +
        <nowiki></source></nowiki>
 +
      </ports>
 +
    </nocblock>
 +
 
 +
 
 +
===GNU Radio Integration===
 +
GNU Radio is built around the concept of blocks, similarly to RFNoC. When mapping RFNoC into an application, the simple constraint is made that every RFNoC block maps to a single GNU Radio block. Thus, when creating mixed GNU Radio/RFNoC applications, there is a very clear 1:1 mapping between what’s happening in RFNoC and GNU Radio.
 +
 
 +
Since most RFNoC blocks behave very similar to one another from GNU Radio’s perspective, it is generally not required to write C++ code for another block. Rather, a default block provided by RFNoC can be used with appropriate configuration. However, in some cases it may be desirable or even necessary to write a custom GNU Radio block for more specific controlling of the underlying RFNoC block. GNU Radio allows writing blocks in either C++ or Python, but since UHD and RFNoC do not have a Python API, a custom wrapper for an RFNoC block needs to be written in C++. RFNoC Modtool will create skeleton files for this purpose.
 +
 
 +
The most popular and effective way to use GNU Radio is through the graphical interface, the GNU Radio Companion (GRC). GRC requires a separate description of every GNU Radio block in order to become available in the graphical UI, and the same is true for an RFNoC block that is wrapped in a GNU Radio block (even if the generic RFNoC block wrapper is used). For GNU Radio 3.7 and earlier, GRC bindings for blocks are written as XML files with interspersed Cheetah or Python statements. For a more detailed tutorial on how to write these files, refer to the [http://gnuradio.org/redmine/projects/gnuradio/wiki GNU Radio Documentation] and associated [http://gnuradio.org/redmine/projects/gnuradio/wiki/Guided_Tutorials tutorials].
 +
 
 +
====GNU Radio Block Code====
 +
 
 +
* C++ or Python, although RFNoC blocks need to be written in C++ (if at all)
 +
* How does GNU Radio interface to RFNoC?
 +
** via C++ infrastructure code in <code>gr-ettus</code>
 +
** <code>gr-ettus</code> provides a base RFNoC block class
 +
** Users extend base class for their RFNoC blocks
 +
** Many blocks can use base class “as is”
 +
** No C++ or Python code!
 +
* <code>rfnoc-tutorial/lib/gain_impl.cc</code>
 +
** The gain block does not need anything additional
 +
 
 +
====GNU Radio Companion Bindings====
 +
* XML
 +
* Describes GNU Radio blocks to GRC
 +
* No recompilation
 +
* Requirement of GNU Radio Companion
 +
* Not strictly necessary for GNU Radio
 +
* Tutorial on how to write them:
 +
** [http://gnuradio.org/redmine/projects/gnuradio/wiki/GNURadioCompanion http://gnuradio.org/redmine/projects/gnuradio/wiki/GNURadioCompanion ]
 +
* Skeleton file generated by RFNoC Modtool
 +
 
 +
 
 +
Open the <code>tutorial-gain.xml</code> file located in the <code>rfnoc-tutorial/grc</code> directory and edit as follows:
 +
 
 +
 
 +
 
 +
    <nowiki><?xml version="1.0"?></nowiki>
 +
    <block>
 +
      <name>RFNoC: gain</name>
 +
      <key>tutorial_gain</key>
 +
      <category>tutorial</category>
 +
      <import>import tutorial</import>
 +
      <make>tutorial.gain(
 +
        self.device3,
 +
        uhd.stream_args( \# TX Stream Args
 +
            cpu_format="'''fc32'''",
 +
            otw_format="'''sc16'''",
 +
            args="gr_vlen={0},{1}".format(${grvlen}, "" if $grvlen == 1 else "spp={0}".format($grvlen)),
 +
        ),
 +
        uhd.stream_args( \# RX Stream Args
 +
            cpu_format="'''fc32'''",
 +
            otw_format="'''sc16'''",
 +
            args="gr_vlen={0},{1}".format(${grvlen}, "" if $grvlen == 1 else "spp={0}".format($grvlen)),
 +
        ),
 +
        $block_index, $device_index,
 +
      )
 +
    '''self.$(id).set_arg("gain", $gain)'''
 +
      '''</make>'''
 +
      '''<callback>set_arg("gain", $gain)</callback>'''
 +
      <nowiki><!-- Make one 'param' node for every Parameter you want settable from the GUI.
 +
          Sub-nodes:
 +
          * name
 +
          * key (makes the value accessible as $keyname, e.g. in the make node)
 +
          * type --></nowiki>
 +
   
 +
        . 
 +
        .
 +
        .
 +
   
 +
        <option>
 +
          <name>Byte</name>
 +
          <key>u8</key>
 +
        </option>
 +
      </param>
 +
      <param>
 +
        <name>'''Gain'''</name>
 +
        <key>'''gain'''</key>
 +
        '''<value>1.0</value>'''
 +
        <type>'''real'''</type>
 +
      </param>
 +
   
 +
      <nowiki><!-- Make one 'sink' node per input. Sub-nodes:
 +
          * name (an identifier for the GUI)
 +
          * type
 +
          * vlen
 +
          * optional (set to 1 for optional inputs) --></nowiki>
 +
      <sink>
 +
        <name>in</name>
 +
        <type>'''complex'''</type>
 +
        <vlen>$grvlen</vlen>
 +
        <domain>rfnoc</domain>
 +
      </sink>
 +
   
 +
      <nowiki><!-- Make one 'source' node per output. Sub-nodes:
 +
          * name (an identifier for the GUI)
 +
          * type
 +
          * vlen
 +
          * optional (set to 1 for optional inputs) --></nowiki>
 +
      <nowiki><source></nowiki>
 +
        <name>out</name>
 +
        <type>'''complex'''</type>
 +
        <vlen>$grvlen</vlen>
 +
        <domain>rfnoc</domain>
 +
      <nowiki></source></nowiki>
 +
    </block>
 +
 
 +
 
 +
'''NOTE:''' Indentation spacing is important in the <code><make></code> section.
 +
 
 +
===Compile, Install and Verify===
 +
 
 +
    $ cd {USER_PREFIX}/src/rfnoc-tutorial/build
 +
    $ make install
 +
   
 +
    $ uhd_usrp_probe
 +
   
 +
    |  |    _____________________________________________________
 +
    |  |    /
 +
    |  |  |      RFNoC blocks on this device:
 +
    |  |  | 
 +
    |  |  |  * DmaFIFO_0
 +
    |  |  |  * Radio_0
 +
    |  |  |  * Radio_1
 +
    |  |  |  * '''gain_0'''
 +
    |  |  |  * DDC_0
 +
    |  |  |  * FFT_0
 +
    |  |  |  * FIFO_0
 +
    |  |  |  * FIFO_1
 +
    |  |  |  * FIFO_2
 +
 
 +
'''NOTE:''' In the case where the <code>gain_0</code> does not appear but <code>Block_0</code> does: Most likely, the XML block declaration file (see [[Getting_Started_with_RFNoC_Development#UHD_integration|UHD Integration]] section) for the block contains a NoC-ID that does not match with any NoC-ID defined in the hardware part of the design. The user has to be certain that the description files are up-to-date and that the NoC-ID matches in the SW and HW side. See the [[Getting_Started_with_RFNoC_Development#UHD_integration|UHD Integration]] section to update those host side files.
 +
 
 +
==Testing out the custom block==
 +
At this point the custom <code>gain</code> RFNoc Block (Computation Engine) can be used within a GNU Radio flowgraph. Below is an example GRC flowgraph using our new block as well as the output application it produces.
 +
 
 +
[[File:rfnoc gsg an 13.png|center|800px|]]
 +
 
 +
'''NOTE:''' Copy Block: In the RFNoC domain, streams of data can not be split as easily as they are in the GNU Radio domain. The “copy” block depicted in the screenshot above serves the function as a stream splitter . It’s main purpose, when “enabled”, is to copy the samples it is getting at its input and putting then into the output, but here it is also serving as a boundary between a RFNoC-domain and a GNURadio-domain. In the flowgraph above. after this boundary is passed, the data stream can easily be split into the two sinks to have them run simultaneously (standard GNU Radio functionality). It is possible to connect the GNU Radio blocks directly to RFNoC blocks without a “copy” block, but only one would work at a time (the other ones would have to be disabled). Another way to split data streams from the RFNoC-domain is to use the ”RFNoC: split stream” block, which would split the streams in the RFNoC domain, but this is not very useful here as we are, in any case, moving into the GNURadio-domain.
 +
 
 +
[[File:rfnoc gsg an 14.png|center|800px|]]
 +
 
 +
==Troubleshooting==
 +
===Xilinx Vivado===
 +
====Compile issues====
 +
=====Synthesis is failing=====
 +
Verify all the correct Xilinx [[Getting Started with RFNoC Development#Prerequisites|prerequisite software]] is installed.
 +
 
 +
Additional helpful information can be found in the following Xilinx forum posts:
 +
* https://forums.xilinx.com/t5/Synthesis/Synthesis-failed-without-reporting-any-error/td-p/686000
 +
* https://forums.xilinx.com/t5/Installation-and-Licensing/Vivado-on-Linux-synthesis-fails-with-no-error-message/td-p/732143
 +
 
 +
====Environment Setup====
 +
The <code>uhd_image_builder.py</code> script will also set up the Xilinx Vivado environment by automatically running the <code>setupenv.sh</code> located in the <code>{USER_PREFIX}/src/uhd-fpga/usrp3/top/{device}</code> directory. The <code>setupenv.sh</code> script assumes that Xilinx Vivado is installed in the default location of <code>/opt/Xilinx/Vivado</code>. If the installation is in a different directory, then the <code>setupenv_base.sh</code> script will need to be modified. The script is located at: <code>{USER_PREFIX}/src/uhd-fpga/usrp3_rfnoc/tools/scripts/setupenv_base.sh</code>.
 +
 
 +
 
 +
==Reference Files==
 +
The following reference files are included within the gain_src.tar.gz archive linked below:
 +
 
 +
* gain.xml
 +
* noc_block_gain.v
 +
* noc_block_gain_tb.sv
 +
* tutorial_gain.xml
 +
* rfnoc_gain.grc
 +
 
 +
[[Media:gain src.tar.gz]]
  
−
= Other Resources =
+
==Links and Additional Resources==
−
[1] [http://gnuradio.org/redmine/projects/gnuradio/wiki/OutOfTreeModules GNURadio OutOfTree Modules tutorial]
+
===RFNoC additional resources===
 +
* [https://youtube.com/watch?v=j-EfyPVpaJ8 Video: RFNoC Getting Started Video Tutorial]
 +
* [https://kb.ettus.com/Mailing_Lists USRP Mailing List]
 +
* [https://kb.ettus.com/RFNoC RFNoC Software Resources Page]
 +
* [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_Intro.pdf RFNoC Introduction]
 +
* [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_FPGA.pdf RFNoC Deep Dive: FPGA]
 +
* [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_Host.pdf RFNoC Deep Dive: Host side]
 +
* [https://www.youtube.com/watch?v=8cPd3t88djE Video: RFNoC presented at Wireless @ Virginia Tech, 2015 ]
 +
** Explaining the slides of Intro, FPGA and Host presentations above (in that order).
 +
* [https://www.youtube.com/watch?v=51rpjJ2W0Qs Video: It's the RFNoC Life for Us by Martin Braun at GRCon16, 2016]
  
−
[2] [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_Intro.pdf Introduction to RFNoC]
+
===GNU Radio resources===
 +
* [http://gnuradio.org/redmine/projects/gnuradio/wiki/OutOfTreeModules GNU Radio OutOfTree Modules tutorial]
 +
* [http://gnuradio.org/redmine/projects/gnuradio/wiki/InstallingGRFromSource GNU Radio Installation]
 +
* [http://gnuradio.org/redmine/projects/gnuradio/wiki/Tutorials GNU Radio Tutorials]
  
−
[3] [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_FPGA.pdf RFNoC Deep Dive: FPGA]
+
===UHD resources===
 +
* [http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com USRP Mailing List]
 +
* [https://kb.ettus.com/UHD UHD Software Resources Page]
 +
* [http://files.ettus.com/manual/md_usrp3_build_instructions.html USRP3 build instructions]
 +
* [http://files.ettus.com/manual/ UHD Manual]
  
−
[4] [https://www.ettus.com/content/files/RFNoC_Wireless_at_VT_Host.pdf RFNoC Deep Dive: Host side]
+
===Other resources===
 +
* [https://www.xilinx.com/support/documentation/ip_documentation/ug761_axi_reference_guide.pdf Xilinx - AXI reference guide]
 +
* [https://kb.ettus.com/Building_and_Installing_the_USRP_Open-Source_Toolchain_(UHD_and_GNU_Radio)_on_Linux UHD + GNU Radio Application Note (Linux)]
 +
* [http://gnuradio.org/redmine/projects/pybombs/wiki PyBOMBS]
  
−
[5] [https://www.youtube.com/watch?v=8cPd3t88djE Video: RFNoC presented at Wireless @ Virginia Tech, 2015] Explaining the slides of [2][3][4] (In that order)
 
  
−
[6] [https://github.com/EttusResearch/uhd/wiki/RFNoC:-Getting-Started GitHub: RFNoC -Getting Started]
 
  
−
[7] [https://github.com/EttusResearch/uhd/wiki/RFNoC:--Specification GitHub: RFNoC - Specification]
 
  
  
−
[[Category:Software Resources]]
+
[[Category:Application Notes]]

Latest revision as of 23:04, 30 November 2023

Application Note Number

AN-823

Abstract

Note: This application note is DEPRECATED. It applies only to using RFNoC in UHD 3.x. The latest Getting Started in RFNoC application note is available here: Getting Started with RFNoC in UHD 4.0.

This application note guides a user through basic information on the RFNoC architecture, installing necessary software to develop custom RFNoC blocks, also called Computation Engines (CE), and walks through the steps of creating a custom RFNoC block using an example. RFNoC is currently supported on any 3rd generation USRP hardware, currently: E310/E312, E320, N300/N310/N320/N321, and X300/X310. However, this document primarily covers using RFNoC for the USRP X300/X310 and E310/E312. Using RFNoC with the other USRPs is similar to that documented herein.

Overview

First sections deal with installing tools and validating correct tool installation in order to do RFNoC development. Later sections deal with creating a custom RFNoC block, using the built-in testbench architecture, building an FPGA image with the custom block and finally testing out the new block within GNU Radio.

Licensing

The RFNoC code base is open source, including code that executes on the host, as well as code targeted to the USRP hardware (FPGA and microcontroller firmware). RFNoC is available under the open-source GNU Lesser General Public License (LGPL). For more information on our licensing policy, please contact info@ettus.com.

Prerequisites

RFNoC is only supported on 3rd generation USRP hardware as noted in the Abstract.

In order to build custom USRP FPGA images and RFNoC blocks the following hardware and software are needed.

  • Ubuntu 14.04.5 or 16.04.1 (preferred): Currently PyBOMBS (which can be used to install the Software build tools), works most reliably in Ubuntu, and thus, we recommend using this distribution. Also, a majority of the scripts used during the build process are Linux (Ubuntu) specific. A PC with multiple cores and 8GB+ of RAM is recommended.
  • Xilinx Vivado tools (version 2017.4): The specific version depends on the branch and state of the FPGA code. The default install location is /opt/Xilinx/Vivado. Once all of the Software build tools are installed the specific version for the downloaded code can be found in the setupenv.sh file located in the {USER_PREFIX}/src/uhd-fpga/usrp3/top/{DEVICE} directory. Further information can be found here.
  • Software build tools: If UHD can be or has been compiled from source on the development PC then all the necessary software build components are present (PyBOMBS can be used to set all this up and instructions on how to do so are given in a following step).
  • Any 3rd generation USRP hardware as noted in the Abstract.

NOTE:

  • The edition of Xilinx Vivado that is required will depend on which USRP device is being used.
    • X3xx series devices: Design Edition or System Edition.
    • E3xx series devices: Design Edition, System Edition, or the free WebPack Edition.
  • Other operating systems can be used, but the exact steps on how to proceed are not given in this Application Note.
  • In some Linux distributions (e.g. Ubuntu) dash is set as default shell, which may cause some issues. It is recommended to set the shell to bash by running the following commands in the terminal. Choose <No> when prompted by the first command and the second command will validate the that bash will be used.
   $ sudo dpkg-reconfigure dash
   $ ll /bin/sh

Creating a development environment

While this Application Note goes through the process of integrating GNU Radio into the RFNoC development flow, it is by no means required to use or develop within the RFNoC framework, but it makes it a great deal easier to use a framework on top of RFNoC for aspects such as visualization and other features. GNU Radio is freely available and more information about it can be found here.

The following software packages are required in order to setup a development environment/sandbox:

  • UHD
  • GNU Radio
  • gr-ettus

Create development environment using PyBOMBS

The cleanest way to set this up is to install everything into a dedicated directory. PyBOMBS is the simplest way to do this. If not already installed, PyBOMBS can be setup with the following commands: 

   $ sudo apt-get install git
   $ sudo apt-get install python-setuptools python-dev python-pip build-essential 
   
   $ sudo pip install git+https://github.com/gnuradio/pybombs.git
   $ pybombs recipes add gr-recipes git+https://github.com/gnuradio/gr-recipes.git
   $ pybombs recipes add ettus git+https://github.com/EttusResearch/ettus-pybombs.git

These commands will do the following:

  • Install Git
  • Install pip and other Python dependencies
  • Install the latest PyBOMBS from its Git repository
  • Add the gr-recipes recipes which are used to install GNU Radio specific software
  • Add the ettus recipes which are used to install Ettus Research specific software

From here, PyBOMBS can be used to setup and install the development environment/sandbox by running the following command: 

   $ pybombs prefix init ~/rfnoc -R rfnoc -a rfnoc

This will do the following:

  • Create a directory in the user’s home directory called rfnoc (any valid directory name will work)
  • Give the prefix an alias of rfnoc ( [-a alias], e.g. –a rfnoc ), which would be the name given to this path. This name will be used in further steps that use PyBOMBS. When creating the first prefix and omitting the alias, the prefix will be setup as the default.
  • Use the rfnoc prefix recipe ( as opposed to a package recipe like gqrx ) to clone UHD, FPGA, GNU Radio, and gr-ettus sources into the ~/rfnoc directory as well as compile and install all the software

NOTE: A user can specify how many cores are used by builds when using PyBOMBS. The default is set to 4. For example, this will set the number of cores used to 3: 

   $ pybombs config makewidth 3

The value will be written into a configuration file and then applied to subsequent PyBOMBS commands. This value can temporarily be overridden for a specific build by specifying the --config makewidth=X argument, where “X” is an integer number. If the user only has 4 cores it is recommend to use this argument in the pybombs command to limit the number of cores to <4 (e.g. 3) so that the computer stays responsive. Following are 2 examples, one using less cores and the other using more cores: 

   $ pybombs --config makewidth=3 prefix init ~/rfnoc -R rfnoc -a rfnoc 
   $ pybombs --config makewidth=7 prefix init ~/rfnoc -R rfnoc -a rfnoc

Then, it is necessary to setup the PyBOMBS environment, so that the system/terminal session will have the environmental variables pointing to this newly created prefix, which is done with the following commands: 

   $ cd ~/rfnoc
   $ source ./setup_env.sh

Once the previous command is run, this terminal session will have access to the environmental variables that allow the complete use of the set of software that was just installed with PyBOMBS. If access to the software is needed in other terminals the same command must be run within them.

NOTE: Throughout the rest of this document the term {USER_PREFIX} will used at the beginning of different directories. For example, {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts is a directory that contains useful scripts for compiling. The term {USER_PREFIX} is used to denote the folders that precede the /src directory. Examples of what {USER_PREFIX} could be: /home/user/rfnoc or /home/user/myDevfolder/. On many Linux environments using ~/ at the beginning of the target directory path is equivalent to the user’s home directory.( i.e ~/ is equal to /home/user/). So {USER_PREFIX} could also look like ~/rfnoc or ~/myDevfolder/.

Create the development environment manually

As an alternative to using PyBOMBS, manually installing and configuring the software is done by following the individual install notes for GNU Radio, UHD and gr-ettus and by making sure they are reachable by linkers and compilers.

NOTE: The Application Note found here goes through the process of manually installing UHD and GNU Radio on Linux platforms.

To manually download the software, use these git clone commands, which will select the correct branches: 

   $ git clone --recursive -b rfnoc-devel https://github.com/EttusResearch/uhd.git 
   $ git clone --recursive -b maint https://github.com/gnuradio/gnuradio.git # master branch is also fine instead of maint
   $ git clone -b master https://github.com/EttusResearch/gr-ettus.git 
   $ git clone -b rfnoc-devel https://github.com/EttusResearch/fpga.git

If UHD, GNU Radio and/or gr-ettus are already installed, it would be sufficient to checkout the branches mentioned and update them them (git pull). Thereafter, rebuild each of the repositories (rebuild order: UHD, GNU Radio, gr-ettus).

Verify Environment

Running the command “uhd_config_info” with the “--version” flag will verify that the installation has been completed successfully.

NOTE: The version string output from this command may differ, however it should be similar to the output below. 

   $ uhd_config_info --version
   linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-161- g83150fdd
   
   4.0.0.rfnoc-devel-161-g83150fdd

Testing the default FPGA image and building from existing blocks

It is recommended to spend a moment looking at the Ettus Research default image, which is pre-built with a set of RFNoC blocks, as well as building a custom image with a unique set of pre-built RFNoC blocks. To get the default image(s), run the following command: 

   $ uhd_images_downloader

Ettus Research will be updating the default image(s) occasionally, and uhd_images_downloader can be run anytime after running git pull and re-installing to pull the most current images. Images are stored in the {USER_PREFIX}/share/uhd/images directory.

The following images have the corresponding RFNoC blocks (Computation Engines):

Image Name Included Blocks
usrp_x300_fpga_HG.bit

usrp_x300_fpga_XG.bit

usrp_x310_fpga_HG.bit

usrp_x310_fpga_XG.bit

2x DDC, 2x DUC
usrp_x300_fpga_RFNOC_HG.bit

usrp_x300_fpga_RFNOC_XG.bit

DUC, DDC (one channel), fosphor, window, fft, 2x AXI FIFOs
usrp_x310_fpga_RFNOC_HG.bit

usrp_x310_fpga_RFNOC_XG.bit

DUC, DDC (one channel), fosphor, window, fft, 2x AXI FIFOs, Keep One in N, FIR, Siggen
usrp_e310_fpga.bit

usrp_e310_fpga_sg3.bit

1x DDC, 1x DUC
usrp_e310_fpga_RFNOC.bit (sg1 version)

usrp_e310_fpga_RFNOC_sg3.bit

fosphor, window, fft, 2x AXI FIFOs, FIR

Instructions on flashing the image to a device can be found here for X3xx series and here for E3xx series.

NOTE: FPGA images are specific to the USRP device NOT the USRP series. For example, a USRP X300 FPGA image will NOT work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device.

By following the steps above the following should now be available:

  • UHD/RFNoC code downloaded and installed
  • FPGA code available
  • A valid RFNoC image on your X3xx or E3xx series device

Inspect default images

Run the following command, with a USRP connected to your PC, to verify current image on the USRP. 

   $ uhd_usrp_probe

If an RFNoC image was successfully loaded onto the USRP, there will be a lot of output text (RFNoC code is currently very verbose). The final lines of the output should be similar to the following for an USRP X310 ( e.g. usrp_x310_fpga_HG ): 

   |   |     _____________________________________________________
   |   |    /
   |   |   |       RFNoC blocks on this device:
   |   |   |   
   |   |   |   * DmaFIFO_0
   |   |   |   * Radio_0
   |   |   |   * Radio_1
   |   |   |   * DDC_0
   |   |   |   * DDC_1
   |   |   |   * DUC_0
   |   |   |   * DUC_1


Final output for usrp_x310_fpga_RFNOC_HG.bit image: 

   |   |     _____________________________________________________
   |   |    /
   |   |   |       RFNoC blocks on this device:
   |   |   |   
   |   |   |   * DmaFIFO_0
   |   |   |   * Radio_0
   |   |   |   * Radio_1
   |   |   |   * DDC_0
   |   |   |   * DUC_0
   |   |   |   * FFT_0
   |   |   |   * Window_0
   |   |   |   * FIR_0
   |   |   |   * SigGen_0
   |   |   |   * KeepOneInN_0
   |   |   |   * fosphor_0
   |   |   |   * FIFO_0
   |   |   |   * FIFO_1


NOTE: The actual names and number of blocks can differ. The list of blocks should start with the DmaFIFO_x and Radio_x, and then a couple more lines of block IDs should follow.

Build custom image with pre-built RFNoC blocks

Because of the growing number of RFNoC blocks, the user has the option to build an FPGA image with a set of pre-built RFNoC blocks of their choosing. The following steps describe the process for doing this and by so doing will also validate proper tool installation. Because compilation can take a couple of hours, it is recommended the user begin this process while continuing the rest of this guide.

NOTE: FPGA compilations can run in the background, however they are very resource intensive. If the user intents to use the same computer that is compiling to walk through the rest of this Application Note, it is recommended that the computer has plenty of resources.

The script to initiate a compile is called uhd_image_builder.py, and is located in the {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts directory. Run the help menu by typing: 

   $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts 
   $ ./uhd_image_builder.py --help

A more detailed discussion of this script is given in an upcoming section. For now, compiling an FPGA image that has 2 RFNoC blocks (window and fft) and some FIFOs, is done by running the script with the following arguments.

Example for an X310 USRP: 

   $ ./uhd_image_builder.py window fft -d x310 -t X310_RFNOC_HG -m 5 --fill-with-fifos

Example for an E310 USRP with Speed Grade 3 (sg3) FPGA: 

   $ ./uhd_image_builder.py window fft -d e310 -t E310_RFNOC_sg3 -m 5 --fill-with-fifos

At the end of a successful compilation process, write the new image to a USRP. If the image was compiled for a USRP X310, the following command will load the new image. Update the {IP_Address} of the USRP and {USER_PREFIX} to the appropriate values for your configuration before running the command. 

   $ uhd_image_loader --args "type=x300,addr={IP_ADDRESS}" --fpga-path {USER_PREFIX}/src/uhd-fpga/usrp3/top/x300/build/usrp_x310_fpga_RFNOC_HG.bit

NOTE: FPGA images are specific to the USRP device NOT the USRP series. For example, a USRP X300 FPGA image will NOT work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device. Additional instructions on flashing a custom image to a device can be found here for X3xx series and here for E3xx series.

After the image has been successfully written to the USRP, power-cycle it and run the “uhd_usrp_probe” utility to view the newly compiled blocks. 

   $ uhd_usrp_probe

The final lines of output for the image built for the X310 is as follows: 

   |   |     _____________________________________________________
   |   |    /
   |   |   |       RFNoC blocks on this device:
   |   |   |   
   |   |   |   * DmaFIFO_0
   |   |   |   * Radio_0
   |   |   |   * Radio_1
   |   |   |   * Window_0
   |   |   |   * FFT_0
   |   |   |   * FIFO_0
   |   |   |   * FIFO_1
   |   |   |   * FIFO_2

Getting started with UHD + RFNoC

The following new examples included within the rfnoc-devel branch of UHD, are a good reference on how to use RFNoC from UHD.

The following example is based off of rx_samples_to_file.cpp. The example can be configured to place an RFNoC block in between the radio and host. 

   rfnoc_rx_to_file.cpp

This next example chains a null source to another block and streams the data to the host. 

   rfnoc_nullsource_ce_rx.cpp

These examples demonstrate the core features and flexibility of RFNoC.

For more information on UHD and UHD development please refer to the UHD Software Resource page, Getting Started with UHD and C++ Application Note or directly to the UHD user manual.

Getting started with GNU Radio + RFNoC

A good way of getting started with RFNoC in a more visual way is to use GNU Radio. The gr-ettus out-of-tree module (OOT) allows a user to use RFNoC blocks in their local GNU Radio / GNU Radio Companion (GRC) installation. This GNU Radio OOT contains blocks that allow you to configure your FPGA through GRC.

NOTE: As blocks in the gr-ettus OOT mature, they will be upstreamed to gr-uhd. Also, gr-ettus is a container used by Ettus Research to disseminate experimental or under-development features for gr-uhd. It is not a replacement for gr-uhd (in fact, the latter is a requirement for gr-ettus).

Examples can be run from gr-ettus/examples/rfnoc, provided that the appropriate RFNoC blocks are compiled into the FPGA image currently running on the USRP.

A couple of rules for building GNU Radio flowgraphs with RFNoC blocks:

  • You always need a Device3 object in your flow graph (it does not get connected, see screenshot below).
  • You should have at least two RFNoC blocks connected together. Going GNU Radio Block -> RFNoC Block -> GNU Radio Block is not recommended (it will work, but with suboptimal performance).

The GNU Radio flowgraph rfnoc_ddc.grc is an example that can be run using the default RFNoC image. Below are screenshots of the flowgraph and what it produces.

rfnoc gsg an 1.png

NOTE: Copy Block: In the RFNoC domain, streams of data can not be split as easily as they are in the GNU Radio domain. The “copy” block depicted in the screenshot above serves the function as a stream splitter. Its main purpose, when “enabled”, is to copy the samples it is getting at its input and put them into the output, but here it is also serving as a boundary between a RFNoC-domain and a GNURadio-domain. In the flowgraph above, after this boundary is passed, the data stream can easily be split into the two sinks to have them run simultaneously (standard GNU Radio functionality). It is possible to connect the GNU Radio blocks directly to RFNoC blocks without a “copy” block, but only one would work at a time (the other ones would have to be disabled). Another way to split data streams from the RFNoC-domain is to use the ”RFNoC: split stream” block, which would split the streams in the RFNoC domain, but this is not very useful here as we are, in any case, moving into the GNURadio-domain.

rfnoc gsg an 2.png

For more information on GNURadio development please refer to the GNURadio user's manual and API.

Starting a custom RFNoC block using RFNoC Modtool

The figure below shows the basic structure of the RFNoC Stack. Corresponding code is needed in each of the three sections in order to build a custom RFNoC block with GNU Radio integration. A tool called RFNoC Modtool was created in order to minimize the effort needed to implement a new RFNoC block. RFNoC Modtool creates a custom GNU Radio OOT module with the basic structure and the necessary files for each of these sections. RFNoC Modtool is currently a part of the GNU Radio OOT module gr-ettus.

rfnoc gsg an 3.png

RFNoC Modtool Utilization

NOTE: Console outputs may vary depending on the version of UHD the user is running. However, functionality should be the same or similar.

Because the RFNoC Modtool has similar functionality to the gr_modtool [ gr_modtool ] provided by GNU Radio, those that have worked with gr_modtool in the past will find the RFNoC Modtool familiar.

To check the usage of the tool, run the following: 

   $ rfnocmodtool help
   linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
   
   Usage:
   rfnocmodtool <command> [options] -- Run <command> with the given options.
   rfnocmodtool help -- Show a list of commands.
   rfnocmodtool help <command> -- Shows the help for a given command. 
   
   List of possible commands:
   
   Name      Aliases          Description
   =====================================================================
   disable   dis              Disable block (comments out CMake entries for files) 
   info      getinfo,inf      Return information about a given module 
   remove    rm,del           Remove block (delete files and remove Makefile entries) 
   makexml   mx               Make XML file for GRC block bindings 
   add       insert           Add block to the out-of-tree module. 
   newmod    nm,create        Create a new out-of-tree module 
   rename    mv               Rename a block in the out-of-tree module.

Creating an RFNoC OOT Module

To start generating an RFNoC OOT module navigate to the source location ( i.e. cd ~/{USER_PREFIX}/src ) and type: 

   $ rfnocmodtool newmod [NAME OF THE MODULE]


Where [NAME OF THE MODULE] is a name the user gives the new module. In the following, a module is created with the name “tutorial”. If the user does not write the name of the module following the newmod command the tool will ask for it interactively. Running this command will create a folder containing the basic folders that you may need for a functional module. 

   $ rfnocmodtool newmod tutorial
   linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
   
   Creating out-of-tree module in ./rfnoc-tutorial... Done.
   Use 'rfnocmodtool add' to add a new block to this currently empty module.

To see what files and directories were created run: 

   $ ls rfnoc-tutorial/
   apps  cmake  CMakeLists.txt  docs  examples  grc  include  lib  MANIFEST.md  python  README.md  rfnoc  swig


In contrast with gr_modtool, this includes a folder called rfnoc, which is where the UHD/FPGA files are located.

Adding custom blocks to OOT Module

In order to add blocks to a module, navigate to the folder just created and use the add command of rfnocmodtool. Continuing with the example above, run the following: 

   $ cd rfnoc-tutorial
   $ rfnocmodtool add [NAME OF THE BLOCK]

For demonstrative purposes, a block named gain will be created. The gain block will multiply samples that pass through it by a constant. As before, if the name is not given, the tool will ask the user for the name. There are several arguments that can be passed to the tool, but running the tool without any of these arguments will give the following interactive parsing output: 

   $ rfnocmodtool add gain
   linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_4.0.0.rfnoc-devel-162-g335a1317
   
   RFNoC module name identified: tutorial
   Block/code identifier: gain
   Enter valid argument list, including default arguments: 
   Block NoC ID (Hexadecimal): 1111222233334444
   Skip Block Controllers Generation? [UHD block ctrl files] [y/N] N
   Skip Block interface files Generation? [GRC block ctrl files] [y/N] N

Hitting enter on each one of the options will take the default values.

The following is a description of the valid argument list items:

  • NoC ID: This ID is a Hexadecimal number which serves as identification between the hardware part and the software part of the design. It can be as long as 16 0-9 A-F digits. If a NoC ID is not provided, it will be set to a random number.
  • Block Controllers Generation: The block controllers are the C++ control that the user can apply to the UHD-part of the design. In these files, the user can add more control over this layer of the design. Depending on the complexity of the block it may be possible to add all necessary control using NoCScript (more details on NoCScript can be found in the section labeled UHD Integration). In this case the cpp/hpp block control files generation are not needed. Default is to generate as their existence will be ignored if not edited.
  • Block Interface: Add more design specific functionality to the design at the GNU Radio interface by generating these block-interface files and adding necessary logic. Depending on the complexity of the block it may be possible to add all necessary control using NoC-Script. In this case the block-interface files are not needed. Default is to generate as their existence will be ignored if not edited.

NOTE: If the user does not intend to use the block controllers or is not sure if they are needed, the presence of them in the design will do no harm. It is recommended to add them. This leaves the possibility to add more functions inside them in a future stage of development.

After finishing the parsing, the following files will be generated/edited: 

   Adding file 'lib/gain_impl.h'...
   Adding file 'lib/gain_impl.cc'...
   Adding file 'include/tutorial/gain.h'...
   Adding file 'include/tutorial/gain_block_ctrl.hpp'...
   Adding file 'lib/gain_block_ctrl_impl.cpp'...
   Editing swig/tutorial_swig.i...
   Adding file 'python/qa_gain.py'...
   Editing python/CMakeLists.txt...
   Adding file 'grc/tutorial_gain.xml'...
   Adding file 'rfnoc/blocks/gain.xml'...
   Adding file 'rfnoc/fpga-src/noc_block_gain.v'...
   rfnoc/testbenches/noc_block_gain_tb folder created
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv'...
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/Makefile'...
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/CMakeLists.txt'...

Creating FPGA portion of custom RFNoC Block

RFNoC FPGA User Interface (API)

RFNoC blocks or Computation Engines (CEs) in the FPGA use a NoC Shell instance to interface with the rest of RFNoC. NoC Shell implements RFNoC's core functionality: packet muxing and demuxing, flow control, and the settings register bus (i.e. write/read control/status registers). The NoC Shell has an interface to the RFNoC AXI stream crossbar and a user interface. NoC Shell AXI stream interfaces expect CHDR packets with a proper header. See the manual for information on CHDR and SID.

NOTE: AXI Stream is an ARM AMBA standard interface. Xilinx has an AXI Reference Guide with more details on this standard.

rfnoc gsg an 4.png

Many designs will want to use an AXI Stream interface with only sample data. However, as stated earlier, the NoC Shell block expects CHDR packets. To ease interfacing user code, the AXI Wrapper block provides the necessary logic to strip and insert the CHDR header, effectively converting packetized sample data into streaming sample data and vice versa. The example RFNoC blocks noc_block_fft.v and noc_block_fir.v show how AXI Wrapper is used to implement existing Xilinx AXI Stream based IP within a computation engine.

NOTE: AXI Wrapper also supports AXI Stream buses for configuration. These buses are driven via the setting register bus and do not have back pressure. They also consume two user register addresses per bus.

The primary user interface consists of four AXI stream interfaces ( tready, tvalid, tlast, tdata ) and a settings register bus ( 8-bit, valid user register addresses: 128-255 ).


AXI Stream signals:

  • m_axis_data_tdata: Input sample data packets
    • Data coming from host or another CE
  • s_axis_data_tdata: Output sample data packets
    • Data going to another CE or host
  • m_axis_data_tready: Input signal to CE
    • Used to notify CE that downstream CE is ready for data
  • s_axis_data_tready: Output signal to CE
    • Used to notify upstream CE that CE is ready for data
  • m_axis_data_tvalid: Input signal to CE
    • Used to indicate upstream CE has valid data
  • s_axis_data_tvalid: Output signal to CE
    • Used to indicate to downstream CE that CE has valid data
  • m_axis_data_tlast: Input signal to CE
    • Used to delimit packets from upstream CE
  • s_axis_data_tlast: Output signal to CE
    • Used to delimit packets to downstream CE
rfnoc gsg an 5.png
rfnoc gsg an 6.png


Settings Bus signals:

  • set_stb: Assert to write set_data to register at set_address
  • set_addr: Register address to set
  • set_data: Data to set
  • rb_data: Data to read back
  • rb_strobe: Assert to read rb_data from register at set_address
rfnoc gsg an 7.png

For the gain example block the following architecture is desired:

rfnoc gsg an 8.png

Open the file rfnoc-tutorial/rfnoc/fpga-src/noc_block_gain.v that contains the RFNoC block skeleton code that was created when the $ rfnocmodtool add gain command was run and modify the following (BOLD indicates changes to the skeleton code). 

   localparam [7:0] SR_GAIN = SR_USER_REG_BASE;
   localparam [7:0] SR_TEST_REG_1 = SR_USER_REG_BASE + 8'd1;
   
   wire [15:0] gain;
   setting_reg #(
     .my_addr(SR_GAIN), .awidth(8), .width(16))
   sr_gain (
     .clk(ce_clk), .rst(ce_rst),
     .strobe(set_stb), .addr(set_addr), .in(set_data), .out(gain), .changed());
   
    always @(posedge ce_clk) begin
       case(rb_addr)
         8'd0 : rb_data <= {48'd0, gain};
         8'd1 : rb_data <= {32'd0, test_reg_1};
         default : rb_data <= 64'h0BADC0DE0BADC0DE;
       endcase
    end
    
    wire [31:0] pipe_in_tdata;
    wire pipe_in_tvalid, pipe_in_tlast;
    wire pipe_in_tready;

    wire [31:0] pipe_out_tdata;
    wire pipe_out_tvalid, pipe_out_tlast;
    wire pipe_out_tready;

    // Adding FIFO to ensure Pipeline
    axi_fifo_flop #(.WIDTH(32+1))
    pipeline0_axi_fifo_flop (
      .clk(ce_clk),
      .reset(ce_rst),
      .clear(clear_tx_seqnum),
      .i_tdata({m_axis_data_tlast,m_axis_data_tdata}),
      .i_tvalid(m_axis_data_tvalid),
      .i_tready(m_axis_data_tready),
      .o_tdata({pipe_in_tlast,pipe_in_tdata}),
      .o_tvalid(pipe_in_tvalid),
      .o_tready(pipe_in_tready));  

    wire [15:0] i = pipe_in_tdata[31:16];
    wire [15:0] q = pipe_in_tdata[15:0];

    wire [31:0] i_mult_gain = i*gain;
    wire [31:0] q_mult_gain = q*gain;

    wire [31:0] mult_gain = {i_mult_gain[15:0], q_mult_gain[15:0]};
    axi_fifo_flop #(.WIDTH(32+1))
    pipeline1_axi_fifo_flop (
      .clk(ce_clk),
      .reset(ce_rst),
      .clear(clear_tx_seqnum),
      .i_tdata({pipe_in_tlast,mult_gain}),
      .i_tvalid(pipe_in_tvalid),
      .i_tready(pipe_in_tready),
      .o_tdata({pipe_out_tlast,pipe_out_tdata}),
      .o_tvalid(pipe_out_tvalid),
      .o_tready(pipe_out_tready));

    /* Output Signals */
    assign pipe_out_tready = s_axis_data_tready;
    assign s_axis_data_tvalid = pipe_out_tvalid;
    assign s_axis_data_tlast  = pipe_out_tlast;
    assign s_axis_data_tdata  = pipe_out_tdata;

The following is a block diagram of the code created by the above Verilog:

gain block diagram v01.png

NOTE: In order to meet timing, FIFO blocks were added to either side of the Multiplication process.

Creating and running HDL testbenches

In order to make the coding iteration process more efficient, it is recommended to create testbenches for all RFNoC blocks before compiling them into the FPGA image. This allows for flaw and/or bug detection early in the design. RFNoC Modtool provides the structure and files ( e.g. noc_block_{USER_BLOCK_NAME}_tb ) for the testbenches of each of the OOT blocks that are added with the $ rfnocmodtool add command.

Below is a figure that shows the general testbench architecture that is created by the RFNoC Modtool. This architecture allows a user to test their custom block in the exact same environment it will be placed in when it is built into the RFNoC architecture. Other benefits of the testbench architecture include:

  • Testing through multiple blocks (e.g. FILTER -> FFT -> AVE)
  • Testing with multiple streams (e.g. RFNoC block ADD/SUB takes 2 streams, one that will have a constant added to it and one that will have a constant subtracted from it)
  • Data transfer abstraction (e.g. RFNoC Sim Lib API calls to tb_streamer.send and tb_streamer.recv which take care of all the AXI stream signaling)
rfnoc gsg an 9.png

NOTE: The noc_block_tb block is an instantiation of the noc_block_export_io that is used in testbenches to communicate to the RFNoC architecture. This makes it possible to talk “RFNoC” to the user’s custom block and as such the custom block has a complete RFNoC experience (signaling, flowcontrol, addressing, etc)

From the Adding custom blocks to OOT Module section where the gain block was initially created, the last files generated were: 

   rfnoc/testbenches/noc_block_gain_tb folder created
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv'...
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/Makefile'...
   Adding file 'rfnoc/testbenches/noc_block_gain_tb/CMakeLists.txt'...


The noc_block_gain_tb is a folder generated to contain all the files related to the test bench of the gain block. Each time a new OOT block is created, a new folder will be generated as well.

Inside of this folder are the following three files:

  • CMakeLists.txt: this is an empty file used, so far, only to increase the scope of the compilers.
  • noc_block_gain_tb.sv: this is a System Verilog file, in which user custom tests are to be located. This is the only file that needs to be modified.
  • Makefile: This file determines the directives that run the simulation.


The noc_block_gain_tb.sv testbench skeleton code creates the following architecture:

testbench arch gain v01.png

Open the file rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv and modify the following lines:

Right under the “Verification” section: 

   initial begin : tb_main
     string s;
     logic [31:0] random_word;
     logic [63:0] readback;
     logic [15:0] gain;

In the “Test 4 -- Write / readback user registers” section: 

   `TEST_CASE_START("Write / readback user registers");
   random_word = $random();
   tb_streamer.write_user_reg(sid_noc_block_gain, noc_block_gain.SR_GAIN, random_word[15:0]);
   tb_streamer.read_user_reg(sid_noc_block_gain, 0, readback);
   $sformat(s, "User register 0 incorrect readback! Expected: %0d, Actual %0d", readback[15:0], random_word[15:0]);
   `ASSERT_ERROR(readback[15:0] == random_word[15:0], s);

In the “Test 5 -- Test sequence” section: 

   `TEST_CASE_START("Test sequence");
   gain = 100;
   tb_streamer.write_user_reg(sid_noc_block_gain, noc_block_gain.SR_GAIN, gain);
   fork
     begin
       cvita_payload_t send_payload;
       for (int i = 0; i < SPP/2; i++) begin
         send_payload.push_back(64'(i));
       end
       tb_streamer.send(send_payload);
     end
     begin
       cvita_payload_t recv_payload;
       cvita_metadata_t md;
       logic [63:0] expected_value;
       tb_streamer.recv(recv_payload,md);
       for (int i = 0; i < SPP/2; i++) begin
         expected_value = i*gain;

Test #4 verifies that we can write and readback the gain value. Test #5 writes to the gain register, sends a sample set in the form of a ramp (1, 2, 3, 4, etc) to the RFNoC gain block and finally reads the values from the gain block and compares them to expected values. The followings steps will allow the user to run this testbench.

From within the rfnoc-tutorial directory, create a build directory and enter it by running: 

   $ mkdir build && cd build/

The next step is to run cmake. If PyBOMBS was used to create the development sandbox, cmake will automatically detect the location of the fpga repository. If PyBOMBS was not used, the user must provide the location of where the fpga repository is installed.

If PyBOMBS used, run: 

   $ cmake ../

If PyBOMBS not used, run: 

   $ cmake [-DUHD_FPGA_DIR=/PATH/TO/FPGA/REPOSITORY] ../

Final output from the $ cmake ../ command: 

   -- Configuring done
   -- Generating done
   -- Build files have been written to: /home/widow/rfnoc/src/rfnoc-tutorial/build

The following command will modify the necessary files and set the correct path to the simulation tools. From now on, every time a new block is added, this command will be run automatically. Remember, only run the following command once for each OOT module (not RFNoC block, but OOT module) created: 

   $ make test_tb
   Scanning dependencies of target test_tb
   Built target test_tb

Testbenches can be executed by running the command: 

   $ make noc_block_[name_of_your_block]_tb

The gain block testbench can be run by running the following command: 

   $ make noc_block_gain_tb

The simulation will start. Final output should look like this: 


   ========================================================
   TESTBENCH STARTED: noc_block_gain
   ========================================================
   [TEST CASE   1] (t=000000000) BEGIN: Wait for Reset...
   [TEST CASE   1] (t=000001002) DONE... Passed
   [TEST CASE   2] (t=000001002) BEGIN: Check NoC ID...
   Read GAIN NOC ID: 1111222233334444
   [TEST CASE   2] (t=000001238) DONE... Passed
   [TEST CASE   3] (t=000001238) BEGIN: Connect RFNoC blocks...
   Connecting noc_block_tb (SID: 1:0) to noc_block_gain (SID: 0:0)
   Connecting noc_block_gain (SID: 0:0) to noc_block_tb (SID: 1:0)
   [TEST CASE   3] (t=000005457) DONE... Passed
   [TEST CASE   4] (t=000005457) BEGIN: Write / readback user registers...
   [TEST CASE   4] (t=000006888) DONE... Passed
   [TEST CASE   5] (t=000006888) BEGIN: Test sequence...
   [TEST CASE   5] (t=000007633) DONE... Passed
   ========================================================
   TESTBENCH FINISHED: noc_block_gain
    - Time elapsed:   7700 ns             
    - Tests Expected: 5
    - Tests Run:      5
    - Tests Passed:   5
   Result: PASSED   
   ========================================================
   $finish called at time : 7700 ns : File "/home/widow/rfnoc/src/rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/noc_block_gain_tb.sv" Line 10
   INFO: [USF-XSim-96] XSim completed. Design snapshot 'noc_block_gain_tb_behav' loaded.
   INFO: [USF-XSim-97] XSim simulation ran for 1000000000us
   launch_simulation: Time (s): cpu = 00:00:10 ; elapsed = 00:00:12 . Memory (MB): peak = 966.387 ; gain = 54.848 ; free physical = 3080 ; free virtual = 29888
   # if [string equal $vivado_mode "batch"] {
   #     puts "BUILDER: Closing project"
   #     close_project
   # } else {
   #     puts "BUILDER: In GUI mode. Leaving project open."
   # }
   BUILDER: Closing project
   ****** Webtalk v2015.4 (64-bit)
     **** SW Build 1412921 on Wed Nov 18 09:44:32 MST 2015
     **** IP Build 1412160 on Tue Nov 17 13:47:24 MST 2015
       ** Copyright 1986-2015 Xilinx, Inc. All Rights Reserved.
   
   source /home/widow/rfnoc/src/rfnoc-tutorial/rfnoc/testbenches/noc_block_gain_tb/xsim_proj/xsim_proj.hw/webtalk/labtool_webtalk.tcl -notrace
   INFO: [Common 17-206] Exiting Webtalk at Tue Jan 10 23:26:20 2017...
   INFO: [Common 17-206] Exiting Vivado at Tue Jan 10 23:26:22 2017...
   Built target noc_block_gain_tb


With every custom block created, a make directive will be available to run the simulation from the build directory.

Building the FPGA image with a custom user block

In this section steps are given on how to initiate an FPGA build while incorporating the user’s custom RFNoC block. The first sections give general information on building RFNoC images. The remaining two sections show how to initiate FPGA builds using a command line interface and using a graphical interface (coming out soon), respectively.

Discussion on number of blocks in an FPGA image

There is a maximum number of blocks that can be added for each device. The maximum amount of computation engines (CEs/RFNoC blocks) that each device can use is 16, but the amount of custom blocks that can be added depends on the device.

If using a device from the X3xx series, from the 16 CEs, there are 6 that will be always added and are not subject to direct customization: 1 CE for the AXI bus, 1 CE for the Ethernet Interface, 2 Radios and 2 Dma FIFOS. Because of this, the application will only allow a number of 10 custom blocks on the X3xx series.

If using a device from the E3xx series, 2 CE engines are always added and are not subject to direct customization: 1 CE for the AXI bus and 1 Radio. This would virtually allow 14 slots for custom blocks. However, given the size of the FPGA on the E3xx series of devices, the application only allows a number of 6 custom blocks.

NOTE: Blocks with higher resource utilization may fill up the FPGA and force the user to include less blocks.

Verify the current maximum values by running the uhd_images_builder.py utility from the scripts directory. 

   $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts
   $ uhd_image_builder.py --help

Discussion on FPGA image targets

RFNoC target names follow the pattern {DEVICE}_RFNOC_{BUILD_TYPE} with the following build types:

  • HG: 1GigE on SFP+ Port0, 10Gig on SFP+ Port
  • XG: 10GigE on both SFP+ ports
  • HLS: Vivado High Level Synthesis enabled
  • sgX: Speed grade for E300 devices (1 or 3)

Some examples are:

  • X310_RFNOC_HG
  • X300_RFNOC_HG
  • X310_RFNOC_XG
  • X300_RFNOC_XG
  • X310_RFNOC_HLS_HG
  • X300_RFNOC_HLS_HG
  • E310_RFNOC (this is for the speed grade 1 FPGA version of E310, append _sg3 for speed grade 3)

NOTE: E310, E312 and E313 all have the same FPGA hardware and therefore will use the E310_RFNOC_{BUILD_TYPE} target. USRP E3xx devices have either sg1 or sg3 hardware, please visit here to find out how to differentiate.

Additional information about the build targets can be found at the build instructions for USRP3, found here.

Image building using the command line

The script uhd_image_builder.py is used to generate the NoC block instantiation file and build the FPGA image. Run the help menu by typing: 

   $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts
   $ ./uhd_image_builder.py --help
        
   usage: uhd_image_builder.py [-h] [-I INCLUDE_DIR [INCLUDE_DIR ...]]
                               [-m MAX_NUM_BLOCKS] [--fill-with-fifos]
                               [-o OUTFILE] [-d DEVICE] [-t TARGET] [-g] [-c]
                               [blocks [blocks ...]]
   
   Generate the NoC block instantiation file
   
   positional arguments:
     blocks                List block names to instantiate.
   
   optional arguments:
     -h, --help            show this help message and exit
     -I INCLUDE_DIR [INCLUDE_DIR ...], --include-dir INCLUDE_DIR [INCLUDE_DIR ...]
                           Path directory of the RFNoC Out-of-Tree module
     -m MAX_NUM_BLOCKS, --max-num-blocks MAX_NUM_BLOCKS
                           Maximum number of blocks (Max. Allowed for x310|x300:
                           10, for e300: 6)
     --fill-with-fifos     If the number of blocks provided was smaller than the
                           max number, fill the rest with FIFOs
     -o OUTFILE, --outfile OUTFILE
                           Output /path/filename - By running this directive, you
                           won't build your IP
     -d DEVICE, --device DEVICE
                           Device to be programmed [x300, x310, e310]
     -t TARGET, --target TARGET
                           Build target - image type [X3X0_RFNOC_HG,
                           X3X0_RFNOC_XG, E310_RFNOC_sg3...]
     -g, --GUI             Open Vivado GUI during the FPGA building process
     -c, --clean-all       Cleans the IP before a new build


Here are details on the usage of the script which is followed by an example:

Blocks: The first arguments are the names of RFNoC blocks that the user wants to have compiled into the new image which are separated by a space. They can be custom blocks from the user’s OOT module or from the ones that are provided from Ettus, or a combination. Blocks provided by Ettus Research are listed (among other sources necessary for the FPGA build) in the {USER_PREFIX}/src/uhd-fpga/usrp3/lib/rfnoc/Makefile.srcs file.

These blocks can be identified by the following pattern: 

   noc_block_{NAME}.v

However, as all the RFNoC blocks have the same noc_block_ prefix, for simplicity this prefix is omitted when listing the blocks in the uhd_image_builder.py utility. As an example of the incorrect and correct way of adding blocks, consider the following examples when adding the noc_block_null_source_sink and noc_block_siggen blocks:

Incorrect method: 

   $ ./uhd_image_builder.py noc_block_null_source_sink noc_block_siggen ...

Correct method: 

   $ ./uhd_image_builder.py null_source_sink siggen ...

NOTE: Blocks generated by the RFNoC Modtool follow the same naming convention.

There is an increasing list of pre-built blocks. Here is a sample:

  • axi_fifo_loopback
  • axi_dma_fifo
  • fir_filter
  • fft
  • null_source_sink
  • schmidl_cox
  • packet_resizer
  • split_stream
  • vector_iir
  • addsub
  • window
  • keep_one_in_n
  • pfb
  • export_io
  • conv_encoder_qpsk
  • siggen
  • logpwr
  • fosphor
  • moving_avg
  • ddc
  • duc


RFNoC related blocks generally reside in fpga/usrp3/lib/rfnoc/.

Block Filename Description
FIFO noc_block_axi_fifo_loopback.v Simple FIFO loopback / passthrough block.
FFT noc_block_fft.v Xilinx coregen based Fast Fourier Transform up to length 4096.
FIR noc_block_fir_filter.v Xilinx coregen based Finite Impulse Response Filter, 41 taps, reconfigurable tap coefficients.
Window noc_block_window.v Windowing block for use with FFT block.
Vector IIR noc_block_vector_iir.v Single pole IIR with configurable coefficients that filters data along vectors (i.e. parallel streams of samples). Useful with FFT output.
Keep One in N noc_block_keep_one_in_n.v Keeps one packet every N packets.
AddSub noc_block_addsub.v Example of using multiple block ports in a single RFNoC block to add and subtract streams.
Null Source Sink noc_block_null_source_sink.v Generates dummy packets and can consume packets at a configurable rate. Useful for testing.
Packet Resizer noc_block_packet_resizer.v Resizes input packets to a configurable size (larger or smaller than source packets).
Split Stream noc_block_split_stream.v Replicates an input stream to a configurable number of output streams.


NOTE: There is a restriction on the amount of blocks that can added into the FPGA image, see the section in this Application Note labeled Discussion on number of blocks in an FPGA image for more information.

-I INCLUDE_DIR: The -I directive provides the path to top OOT directory, which contains the users rfnoc/fpga-src directory which contains the custom blocks. This path is needed by the Xilinx Vivado tool. Inside the fpga-src directory there is a file called Makefile.srcs that contains the path of the OOT module and a list of all the custom OOT blocks. This is an auto generated file, which is amended every time a new block is added to the OOT module. Manually modifying this file is not recommended. If there are multiple OOT modules with various custom blocks that reside in different directories the way to include them all is by separating the different paths by a space (e.g. -I /first/OOT/path/ /second/OOT/path/).

IMPORTANT: Please be sure to terminate the path of your OOT with the "/" character. Otherwise the path might not be recognized.

-d DEVICE: The -d directive directs the script on which USRP device the build is for. If no –d is included the default is –d x310. Generation-3 USRPs and above all support RFNoC.

-t TARGET: The –t directive directs the script on which type of image to build for the chosen device. With each USRP device there are several build options to choose from. Detailed information about the build targets can be found at the build instructions for USRP3, found here. If -t is not included, a default target will be chosen for the given device. For example, the default X310_RFNOC_HG target builds for the –d x310 device. More details on targets can be found in the section of this Application Note labeled Discussion on FPGA image targets.

-m MAX_NUM_BLOCKS: The –m directive specifies the max number of RFNoC blocks to build on the FPGA image. An RFNoC image does not need to fill all available slots with RFNoC blocks.

--fill-with-fifos: The --fill-with-fifos directive will fill the empty RFNoC block slots with FIFOS. As an example, if a user indicates three RFNoC blocks by name and also specifies –m 5 then the other two slots will be filed with FIFOs.

-o OUTFILE: With the -o directive, the RFNoC blocks instantiation file is generated and saved at the desired path with the given name for the user to inspect. The FPGA image will NOT build if this directive is provided. The purpose of the uhd_image_builder.py script is to auto generate an instantiation file and populate the source files needed for the Xilinx Vivado tool to build the FPGA image, however, it may be desirable to only see the effect of adding a custom OOT module in the fpga/ directory, or for inspecting the instantiation file. When the directive is not provided the rfnoc_ce_auto_inst_x3x0.v file is overwritten and the FPGA image build process will start automatically (standard use).

-g, --GUI: Open Vivado GUI during the FPGA building process

-c, --clean-all: Cleans the IP before a new build

Here is how to create an X310 FPGA image incorporating the gain block that was created earlier in this Application Note: 

   $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts     
   $ ./uhd_image_builder.py gain ddc fft -I {USER_PREFIX}/src/rfnoc-tutorial/ -d x310 -t X310_RFNOC_HG -m 6 --fill-with-fifos


At the end of a successful compilation process, write the new image to a USRP. The following command will load the new image. Update the {IP_Address} of the USRP and {USER_PREFIX} to the appropriate values for your configuration before running the command. 

   $ uhd_image_loader --args "type=x300,addr={IP_ADDRESS}" --fpga-path {USER_PREFIX}/src/uhd-fpga/usrp3/top/x300/build/usrp_x310_fpga_RFNOC_HG.bit

NOTE:

  • The FPGA image building process may take over an hour.
  • FPGA images are specific to the USRP device NOT the USRP series. For example, a USRP X300 FPGA image will NOT work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device. Additional instructions on flashing a custom image to a device can be found here for X3xx series and here for E3xx series.
  • [Environment setup] - The uhd_image_builder.py script will also set up the Xilinx Vivado environment by automatically running the setupenv.sh script located in the {USER_PREFIX}/src/uhd-fpga/usrp3/top/{device} directory. The setupenv.sh script assumes that Xilinx Vivado is installed in the default location of /opt/Xilinx/Vivado. If the installation is in a different directory the setupenv_base.sh script will need to be modified. The script is located at: {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts/setupenv_base.sh

Besides the custom gain block, a DDC and FFT block are also being added along with three FIFOs. The DDC, FFT and FIFO blocks are already in the script's path and therefore do not need their path specified (they ship with the Ettus Research FPGA code). The reason three FIFOs are added is because the max number of blocks was specified to be 6 ( -m 6 ) and since only 3 blocks were specifically named the other three slots are filled with FIFOs.


rfnoc gsg an 10.png

NOTE: Instructions on flashing the image to a device can be found here for X3xx series and here for E3xx series. FPGA images are specific to the USRP device NOT the USRP series. For example, a USRP X300 FPGA image will NOT work on a USRP X310 and vice versa. Loading an image that does not correspond to a USRP device will likely brick the device.

Once the newly compiled image is loaded onto a USRP X3xx running the following command will show what RFNoC blocks are available on the FPGA: 

   $ uhd_usrp_probe 
   
   
   |   |     _____________________________________________________
   |   |    /
   |   |   |       RFNoC blocks on this device:
   |   |   |   
   |   |   |   * DmaFIFO_0
   |   |   |   * Radio_0
   |   |   |   * Radio_1
   |   |   |   * Block_0
   |   |   |   * DDC_0
   |   |   |   * FFT_0
   |   |   |   * FIFO_0
   |   |   |   * FIFO_1
   |   |   |   * FIFO_2


NOTE: The reason the custom block is called Block_0 and not gain_0 is because there is still host side software/files that need updated in order for this block to populate it’s proper name. A following section (UHD Integration) will step through the process of updating those host side files.

Using a graphical interface

A graphical user interface for FPGA generation and building is shipped along with the uhd_image_builder.py script. This intuitive application aids in setting up a custom FPGA build.

This utility is located in the same scripts directory as uhd_image_builder.py.

To run it, enter the following commands: 

   $ cd {USER_PREFIX}/src/uhd-fpga/usrp3/tools/scripts/
   $ ./uhd_image_builder_gui

The application will then be launched:

rfnoc gsg an 11.png

1. Select build target: In this panel the available build targets are listed. This list may vary depending on which branch of the FPGA repository this user is using. Only RFNoC targets are listed. The build type descriptions are:

  • HG: 1GigE on SFP+ Port0, 10Gig on SFP+ Port1
  • XG: 10GigE on both SFP+ ports
  • HLS: Vivado High Level Synthesis enabled
  • sgX: Speed grade for E300 devices (1 or 3)

2. List of blocks available: In this panel the available blocks are listed that can be included into a custom design. This list separates the RFNoC blocks provided by Ettus Research and the OOT modules and corresponding blocks that the user adds. Given the hardware differences between the X3xx and E3xx devices, this list will dynamically change when a different device is selected from the panel on the left. This implies that it is necessary to add the OOT modules for each device independently. This is accomplished by using the Add OOT Blocks feature of the application, details of which are explained at #7 (Add OOT Blocks).

3. Blocks in current design: This section gives information on the MAX number of blocks for a given USRP (based on the target selection). There is a maximum number of blocks that can be added for each device. See the section in this App Note labeled "Discussion on number of blocks in an FPGA image" for more information.

4. Blocks in current design: This panel will be populated by adding elements from the available blocks. All the blocks listed in here will be compiled into the FPGA custom image. There is a maximum number of blocks that can be added for each device. See the section in this App Note labeled "Discussion on number of blocks in an FPGA image" for more information.

5. Add button (>>): Manually add the blocks from the central panel into your design.

6. Remove button (<<): Remove blocks from the current design (far-left panel)

7. Fill with FIFOs: By checking this box, the design will fill any available/unspecified block slots with FIFOs. The number of FIFO blocks that will be instantiated is based on the rules of amount of blocks explained at #3. When less than the max amount of blocks are needed for certain implementation, many users choose to fill their design with FIFO blocks.

8. Open Vivado GUI: Open Vivado GUI during the FPGA building process. This allows the user to save a Vivado project with all IP and work within the Vivado GUI for development.

9. Clean IP: Cleans the IP before a new build (recompiles all IP).

10. Add OOT blocks: Manually add RFNoC Modtool-generated OOT modules by pointing the application to the Makefile.srcs file, which is located in the {USER_PREFIX}/src/{USER-OOT-moddir}/rfnoc/fpga-srcs/ directory. After adding this file, blocks will appear under “OOT blocks for XXXX devices”

11. Show Instantiation File: The application auto-generates the instantiation file that is going to be used by Vivado to build the FPGA image. This instantiation file can be viewed and edited before starting the build by clicking this button.

12. Import from GRC: If the user has a GNU Radio flowgraph with RFNoC blocks already in it, this application can read what RFNoC blocks are in the flowgraph and populate the Blocks in current design section of the application with the necessary RFNoC blocks. NOTE: All RFNoC blocks pulled from a .grc file must be in the of List of blocks available before beginning the build.

13. Generate .bit file: Start the build by clicking this button.

14. uhd_image_builder command: The command line command with arguments is dynamically build here as the user selects different options. The user could save this command to use next time they build/compile an FPGA image to avoid having to select all options again.

NOTE: See the latter end of the previous section for additional information on what to expect once the compile has started as well as final output.

Creating Software/Host portion of custom RFNoC Block

Now that the FPGA portion is complete the next step is to add software integration to UHD and GNU Radio as depicted in the RFNoC Stack below.

rfnoc gsg an 12.png

UHD integration

Despite the data processing happening on the FPGA, the host software still has a lot of responsibilities in order for an RFNoC application to function. For example, it needs to know which settings registers are available within an RFNoC block, or what kind of input and output a block has. All of this information goes into the Block Declaration, which is an XML file that is readable by UHD. Often, some simple logic needs to be embedded in the XML file, which we can do by using a simple scripting language called Noc-Script. Changes to the block declaration file are immediately imported into UHD every time an application is executed, and therefore, no software development toolchain needs to be set up.

The list of things declared by the block declaration file includes:

  • Block name and Noc-ID
  • Registers
  • Inputs and outputs (including types)

In some cases, additional C++ code is required to properly control a block from software. In this case, a Block Controller file is required as well as the declaration file. In most cases, the default block controller provided by UHD is sufficient, so no C++ code needs to be written. Writing custom block controllers requires more effort, and means having to set up a programming toolchain. A common reason to write custom C++ block controllers is if setting a register requires a lot of computation, which is not feasible to do within a block declaration file (e.g., using Noc-Script).

Skeleton code for both the block declaration and the block controller (if required) can be generated through RFNoC Modtool.

Because the gain block does not require anything other than simply reading and writing to a single register the default block controller will suffice for this example. However, we will need to add information about the register.

Open the gain.xml file located in the /rfnoc-tutorial/rfnoc/blocks directory and add the following: 


   <?xml version="1.0"?>
   <!--Default XML file-->
   <nocblock>
     <name>gain</name>
     <blockname>gain</blockname>
     <ids>
       <id revision="0">1111222233334444</id>
     </ids>
     <!-- Registers -->
     <registers>
       <setreg>
         <name>GAIN</name>
         <address>128</address>
       </setreg>
     </registers>
     <!-- Args -->
     <args>
       <arg>
         <name>gain</name>
         <type>double</type>
         <value>1.0</value>
         <check>GE($gain, 0.0) AND LE($gain, 32767.0)</check>
         <check_message>Invalid gain.</check_message>
         <action>
           SR_WRITE("GAIN", IROUND($gain))
         </action>
       </arg>
     </args>
     <!--One input, one output. If this is used, better have all the info the C++ file.-->
     <ports>
       <sink>
         <name>in0</name>
         <type>sc16</type>
       </sink>
       <source>
         <name>out0</name>
         <type>sc16</type>
       </source>
     </ports>
   </nocblock>


GNU Radio Integration

GNU Radio is built around the concept of blocks, similarly to RFNoC. When mapping RFNoC into an application, the simple constraint is made that every RFNoC block maps to a single GNU Radio block. Thus, when creating mixed GNU Radio/RFNoC applications, there is a very clear 1:1 mapping between what’s happening in RFNoC and GNU Radio.

Since most RFNoC blocks behave very similar to one another from GNU Radio’s perspective, it is generally not required to write C++ code for another block. Rather, a default block provided by RFNoC can be used with appropriate configuration. However, in some cases it may be desirable or even necessary to write a custom GNU Radio block for more specific controlling of the underlying RFNoC block. GNU Radio allows writing blocks in either C++ or Python, but since UHD and RFNoC do not have a Python API, a custom wrapper for an RFNoC block needs to be written in C++. RFNoC Modtool will create skeleton files for this purpose.

The most popular and effective way to use GNU Radio is through the graphical interface, the GNU Radio Companion (GRC). GRC requires a separate description of every GNU Radio block in order to become available in the graphical UI, and the same is true for an RFNoC block that is wrapped in a GNU Radio block (even if the generic RFNoC block wrapper is used). For GNU Radio 3.7 and earlier, GRC bindings for blocks are written as XML files with interspersed Cheetah or Python statements. For a more detailed tutorial on how to write these files, refer to the GNU Radio Documentation and associated tutorials.

GNU Radio Block Code

  • C++ or Python, although RFNoC blocks need to be written in C++ (if at all)
  • How does GNU Radio interface to RFNoC?
    • via C++ infrastructure code in gr-ettus
    • gr-ettus provides a base RFNoC block class
    • Users extend base class for their RFNoC blocks
    • Many blocks can use base class “as is”
    • No C++ or Python code!
  • rfnoc-tutorial/lib/gain_impl.cc
    • The gain block does not need anything additional

GNU Radio Companion Bindings


Open the tutorial-gain.xml file located in the rfnoc-tutorial/grc directory and edit as follows: 


   <?xml version="1.0"?>
   <block>
     <name>RFNoC: gain</name>
     <key>tutorial_gain</key>
     <category>tutorial</category>
     <import>import tutorial</import>
     <make>tutorial.gain(
       self.device3,
       uhd.stream_args( \# TX Stream Args
           cpu_format="fc32",
           otw_format="sc16",
           args="gr_vlen={0},{1}".format(${grvlen}, "" if $grvlen == 1 else "spp={0}".format($grvlen)),
       ),
       uhd.stream_args( \# RX Stream Args
           cpu_format="fc32",
           otw_format="sc16",
           args="gr_vlen={0},{1}".format(${grvlen}, "" if $grvlen == 1 else "spp={0}".format($grvlen)),
       ),
       $block_index, $device_index,
     )
   self.$(id).set_arg("gain", $gain)
     </make>
     <callback>set_arg("gain", $gain)</callback>
     <!-- Make one 'param' node for every Parameter you want settable from the GUI.
           Sub-nodes:
           * name
           * key (makes the value accessible as $keyname, e.g. in the make node)
           * type -->
   
        .  
        .
        .
   
       <option>
         <name>Byte</name>
         <key>u8</key>
       </option>
     </param>
     <param>
       <name>Gain</name>
       <key>gain</key>
       <value>1.0</value>
       <type>real</type>
     </param>
   
     <!-- Make one 'sink' node per input. Sub-nodes:
           * name (an identifier for the GUI)
           * type
           * vlen
           * optional (set to 1 for optional inputs) -->
     <sink>
       <name>in</name>
       <type>complex</type>
       <vlen>$grvlen</vlen>
       <domain>rfnoc</domain>
     </sink>
   
     <!-- Make one 'source' node per output. Sub-nodes:
           * name (an identifier for the GUI)
           * type
           * vlen
           * optional (set to 1 for optional inputs) -->
     <source>
       <name>out</name>
       <type>complex</type>
       <vlen>$grvlen</vlen>
       <domain>rfnoc</domain>
     </source>
   </block>


NOTE: Indentation spacing is important in the <make> section.

Compile, Install and Verify

   $ cd {USER_PREFIX}/src/rfnoc-tutorial/build
   $ make install
   
   $ uhd_usrp_probe 
   
   |   |     _____________________________________________________
   |   |    /
   |   |   |       RFNoC blocks on this device:
   |   |   |   
   |   |   |   * DmaFIFO_0
   |   |   |   * Radio_0
   |   |   |   * Radio_1
   |   |   |   * gain_0
   |   |   |   * DDC_0
   |   |   |   * FFT_0
   |   |   |   * FIFO_0
   |   |   |   * FIFO_1
   |   |   |   * FIFO_2

NOTE: In the case where the gain_0 does not appear but Block_0 does: Most likely, the XML block declaration file (see UHD Integration section) for the block contains a NoC-ID that does not match with any NoC-ID defined in the hardware part of the design. The user has to be certain that the description files are up-to-date and that the NoC-ID matches in the SW and HW side. See the UHD Integration section to update those host side files.

Testing out the custom block

At this point the custom gain RFNoc Block (Computation Engine) can be used within a GNU Radio flowgraph. Below is an example GRC flowgraph using our new block as well as the output application it produces.

rfnoc gsg an 13.png

NOTE: Copy Block: In the RFNoC domain, streams of data can not be split as easily as they are in the GNU Radio domain. The “copy” block depicted in the screenshot above serves the function as a stream splitter . It’s main purpose, when “enabled”, is to copy the samples it is getting at its input and putting then into the output, but here it is also serving as a boundary between a RFNoC-domain and a GNURadio-domain. In the flowgraph above. after this boundary is passed, the data stream can easily be split into the two sinks to have them run simultaneously (standard GNU Radio functionality). It is possible to connect the GNU Radio blocks directly to RFNoC blocks without a “copy” block, but only one would work at a time (the other ones would have to be disabled). Another way to split data streams from the RFNoC-domain is to use the ”RFNoC: split stream” block, which would split the streams in the RFNoC domain, but this is not very useful here as we are, in any case, moving into the GNURadio-domain.

rfnoc gsg an 14.png

Troubleshooting

Xilinx Vivado

Compile issues

Synthesis is failing

Verify all the correct Xilinx prerequisite software is installed.

Additional helpful information can be found in the following Xilinx forum posts:

Environment Setup

The uhd_image_builder.py script will also set up the Xilinx Vivado environment by automatically running the setupenv.sh located in the {USER_PREFIX}/src/uhd-fpga/usrp3/top/{device} directory. The setupenv.sh script assumes that Xilinx Vivado is installed in the default location of /opt/Xilinx/Vivado. If the installation is in a different directory, then the setupenv_base.sh script will need to be modified. The script is located at: {USER_PREFIX}/src/uhd-fpga/usrp3_rfnoc/tools/scripts/setupenv_base.sh.


Reference Files

The following reference files are included within the gain_src.tar.gz archive linked below:

  • gain.xml
  • noc_block_gain.v
  • noc_block_gain_tb.sv
  • tutorial_gain.xml
  • rfnoc_gain.grc

Media:gain src.tar.gz

Links and Additional Resources

RFNoC additional resources

GNU Radio resources

UHD resources

Other resources

Retrieved from "https://kb.ettus.com/index.php?title=Getting_Started_with_RFNoC_Development&oldid=5919"