Debugging FPGA images

Application Note Number

AN-XXX

Revision History

Date	Author	Details
2016-11-28	Nicolas Cuervo	Initial creation

Abstract

This application note covers the basics to get you through the process of probing the signals inside an FPGA. In order to accomplish that, we will review briefly the 'Xilinx ChipScope Analyzer' and will apply it to one of our core RFNoC blocks: the RFNoC Signal generator. The contents of this AN could suit most of your needs while setting your debug bitstream for a RFNoC design. However, keep in mind that the topics described here are strictly related to Xilinx framework. For further information please refer to Xilinx documentation [1][2].

Overview

When you are developing your own application, you might come to the point on which you would like to build an FPGA image for your USRP. You might want to modify part of the cores, add some custom functionality, or even add your custom RFNoC block! For that you might follow tutorials such as the Building the FPGA image section of one our "getting started" guides.

But how about debugging your HDL code? This comes really handy when you want to follow closely the behavior of your signals within your hardware design. This Application Note will follow the basic steps needed to create a "chipscope image", which allow you to use the Vivado GUI visual tools to debug your design.

Before we start, this App note assumes that you have been working already with some fpga code and you want to debug it. Being this the case, we assume that you have UHD installed, the FPGA repository cloned, the right version of Xilinx Vivado installed (by the moment this is being written we use Vivado 2015.4) and its environment initialized. If not, we assume you are familiar on how to do the previously noted procedures.

For illustration purposes, here we are going to check the status of some of the output signals of one of the RFNoC blocks we currently provide. However, the same procedure can be used to check the status of any signal within your hardware code, being input, output, or intermediate signal, and being the code a core description, a module for your library or your custom RFNoC block.

Note: Keep in mind that this procedure intends to probe the signals in a fully designed block, which has been also built into a FPGA .bit file and is running in a supported device. This is *not* intended to be a way to test directly your designed code, as building an FPGA image may take several minutes (even hours). For small functionality checks, we strongly recommend you to write a testbench for your code, which will allow you to have more iterations without the need of building and synthesizing your hardware description. You can follow the - Writing Testbenches for RFNoC application note to have insights on how to write your own testbench.

Prerequisites

• Vivado (version 2015.4): As stated in the overview, you'll be working directly with HDL code that you need to build and synthesize. Depending on your target device, you may even need a non-free license (which is the case for the X3XX devices). In the case of Ettus' embedded devices, you can proceed with your design using the Vivado Webpack.

Choosing your signals

At this point we assume that you have a verilog code that has been properly tested by the means of simulation/testbench, but that you want to inspect into its functionality deeper by probing its signals while it is running on a device. This could be helpful for many reasons, such as getting a deeper understanding on the state of your signal during a given transaction, which could give you an insight on how it is working (or even, can give you a lead on why it isn't!)

For this AN, we will use out block RFNoC: Signal Generator as our Unit Under Test (UUT). However, all procedures to be done can be easily transfered to your own design. In addition, as most of our block and FPGA code is written in Verilog, we will use it also in this document. So let's get started.

The Signal generator's code can be found under {fpga-repository}/usrp3/lib/rfnoc/. In some of our latest code releases (such as 3.10.0.0), this code is found under {fpga-repository}/usrp3_rfnoc/lib/rfnoc/. In this directory you can find the RFNoC related code that is used in the RFNoC framework. Consequently, all the HDL for the NoC blocks the we provide is located here. Now, open the file noc_block_siggen.v in your IDE or text editor of preference and give it a quick look.

As you can see, the code is not too extensive and is the comments divide it properly based on functionality. If your design is RFNoCModtool-generated, you'll get a similar preliminary structure in the verilog files for your block. For information on how to use RFNoCModtool please refer to RFNoC Development - Getting Started Guide. Normally for rather simple designs you won't have to deal with the RFNoC Shell or the AXI Wrapper configuration. However, for illustration purposes, we are going to take some of the signals from this part of the code and probe them in our debugging process. A total of 11 signals will be selected, each from a different internal stage.

Lets take a look at how this boundaries look like in the FPGA Internals. Each full RFNoC design can include several different blocks, which are also called "computation engines". The picture on the right [3] portraits the computation engine internals in a quite self explanatory fashion, although a slightly more detailed explanation about each of the internals from a Computation Engine can be found at the RFNoC Software Page:

From NoC Shell: At the top of the figure you can see the AXI Crossbar, were all the computation engines are wired up together. This is not part of our UUT in particular. However, the connection between the crossbar and our UUT - NoC Shell can be tested within our code. From here we are taking the set_data/addr/stb, which are readback registers and provide information from this interface.

• set_data
• set_addr
• set_stb

From the AXI Wrapper: our next stage from where we are taking signals is the AXI Wrapper. This can be understood as a translating stage in which the data that goes from and to the user's design is correctly encapsulated into a CHDR packet [5]. By probing this signals we expect to find out that the data that is being transported is correct, and that the transaction also takes places at the right moment.

• s_axis_data_tdata
• s_axis_data_tuser
• s_axis_data_tlast
• s_axis_data_tvalid
• s_axis_data_tready

From the signal generator design: Last but not least, we are probing signals from the UUT IP, which means that we are checking directly the value that certain lines inside the FPGA have the correct value at a certain time. In this case, we'll be checking if the wave type is according with the one selected from the host, that the gain value is propagated correctly and, clearly, if the block is generating signals when it is enable and when it isn't.

• gain
• wave_type
• enable

Setting up the code for ChipScoping

To let know Vivado that we want to probe signals, we have to go directly into the code and mark this signals for debugging. This can be done by using reserved words that describe the synthesizing attributes for a given signal. There is a variety of different attributes that you can give to any signal of your design [2], but here we are going to discuss the ones that serve most of the debugging needs:

• KEEP: This attribute prevents the signal to be optimized or absorbed into logic blocks, which would mean that the signal, even though it would be operational after synthesis, may not be accessible for probing. An example of the syntax for this attribute is as follows:

 VERILOG:
   (* keep = "true" *) wire signal_name;
   assign signal_name = in1 & in2;

 VHDL:
   signal signal_name : std_logic;
   attribute keep : string;
   attribute keep of signal_name : signal is "true";
   signal_name <= in1 and in2;

• KEEP_HIERARCHY: As well as KEEP, this attribute prevents the optimization. However, this attribute can be applied to a module or instance. By using this attribute, the synthesis tools keep the boundary on this signal static. Example:

 VERILOG
   On Module:
   (* keep_hierarchy = "yes" *) module example (in1, in2, out1, out2);
   On Instance:
   (* keep_hierarchy = "yes" *) example e0 (.in1(in1), .in2(in2), .out1(out1));

 VHDL
   On Module:
   attribute keep_hierarchy : string;
   attribute keep_hierarchy of example : architecture is "yes";
   On Instance:
   attribute keep_hierarchy : string;
   attribute keep_hierarchy of e0 : label is "yes";

• DONT_TOUCH: this attribute works just as KEEP and KEEP_HIERARCHY, with the difference that this one is forward-annotated to place and route to prevent logic optimization. In case where other attributes get into conflict with DONT_TOUCH, DONT_TOUCH takes precedence and will be applied. It also can take values yes/no and true/false. Example:

 VERILOG WIRE
   (* dont_touch = "yes" *) wire signal1;
   assign signal1 = in1 & in2;

 VERILOG MODULE
   (* DONT_TOUCH = "yes" *)
   module example (clk, in1, in2, out1);

 VHDL EXAMPLE
   signal sig1 : std_logic;
   attribute dont_touch : string;
   attribute dont_touch of sig1 : signal is "true"

• MARK_DEBUG: This is arguably the most important attribute for our current use case, because it is the one that tells Vivado which nets are going to be debugged. This also prevents optimization over the signal, and in addition prepares it to be probed during operation. Virtually this attribute could be applied to any net within the design, but there are some nets with specific properties could have protection against visibility, and can not be probed. The values for MARK_DEBUG are TRUE/FALSE. Example:

 VERILOG
   (* MARK_DEBUG = "TRUE" *) wire debug_wire;
 VHDL
   attribute MARK_DEBUG : string;
   attribute MARK_DEBUG of signal_name : signal is "TRUE";

For other attributes and options, please refer to Xilinx's documentation [1][2]. We are going to use the given almost in every case, if not always. Now, the syntax is rather simple and so is the applications to the attributes to the code. The resulting file should look as the picture on the right. When you have modified the code, you are ready to build your debug bitstream.

Adding attributes to signals to probe.

Building the debug bitstream

Save the project and finish Synthesis

For this test in particular you are going to need to have a DDC (Digital down converter) in addition to the UUT for visualization purposes on a host. To add this blocks into the bitstream, go to {fpga-repository}/[usrp3|usrp3_rfnoc/tools/scripts/ and inside that directory run the following command:

   $ ./make.py siggen ddc -g

This will set up your Vivado environment and start the build of an FPGA image with the signal generator and the DDC blocks. The option '-g' is telling the script that at some point during the build process the Vivado User Interface should be opened, as it is where we are going to set up our debug image. For make.py usage and options please refer to the Wiring up computation engines and building the FPGA image section of our getting started guide, or simply run make.py --help in your terminal

Note: The FPGA image building process may take over an hour.

The Vivado GUI is going to come up at some point of the synthesis. Right after the Vivado GUI has opened, you can go ahead and cancel the process that is running, which is usually the last part of the synthesis (when it shows 90% done is a safe moment to cancel. See "Saving the project" figure). This is because we first have to set up the parameters for debugging and the synthesis has to be re-run. After canceling, save the project and give it a name of your choice; we are giving here the name AN_chipscope, but you can name the project whatever you like. Right after saving the project, click on 'Run Synthesis', which can be found on the left panel under Project Manager->Synthesis->Run Synthesis.

Saving the Vivado Project

Note: Most of the time Vivado will auto-detect your highest hierarchy module, but it may happen that it just slips to it and then it will ask you which it. If this happens, you can select the verilog file according to the target device that you are chipscoping as the top module (e.g. x300.v or e300.v)

Now wait until the synthesis is finished. This won't take long, and after it finishes a window will prompted saying that the synthesis is done, and asking if you want to run the implementation. Click on cancel, as we need to setup the debugging parameters first.

Setup debug

Go to Project Manager -> Synthesis -> Open Synthesized Design -> Set Up debug, and the wizard will start. Click on next until you see a window listing the nets to debug. Here, two scenarios are expected. See the figure below:

Sometimes Vivado will pick up the clock domain automatically (which is the case depicted on the right side), but there are occasions where the nets to debug aren't clearly defined under a clock domain and this cases require a little bit more of work, depending on your knowledge of your design. In this case, we know that the nets are under the same clock domain as the other signals, but in case of doubt, you'll have to go and find it out through the code. The case on the left can be solved easily by clicking on 'more info', which is just at the end of the red warning. Right after clickling, further, clearly, information will appear. In the prompted dialog, let us click on 'Assign All Clock Domains'

A window will appear where you can choose the common clock domain on which you want to have the signals. Here, in our case, we select radio_clk_gen/inst/CLK_OUT1, and that would be sufficient to continue. Accept and click next.

Right after, we have to choose the Integrated Logic Analizer - ILA Core Options [6][7]. Here we will only focus on "Sample of data depth" and "Input pipe stages"

The "Sample of data depth" is the maximum number of data sample words that the ILA core can store at run time for each of the probe lines. The input pipe stages is the number of flops or registers that are added to each probe line. This basically determines how big the debug setup will be, and the amount of data that is going to be analyzed per run. Here we select 4096 for the data depth and 1 input pipe line. For further information about the ILA and its configuration, please refer to the ILA documentation [6][7]. With this, the set up is done, and you can proceed to click "next" and then "finish" to complete the wizard operation.

After the setup is finished, go to the left panel and click Project Manager->Program and Debug->Generate Bitstream. This will ask you if you want to run the implementation first, to what we answer 'Yes'. This will prepare the bitstream with which we are going to program our device and debug our design.

Running the debug bitstream in the target device

 #TODO

External references

[1] Vivado Tutorial: Programming and debugging

[2] Vivado Synthesis

[3] RFNoC Deep Dive: FPGA

[4] Xilinx - AXI reference guide

[5] Radio Transport Protocols

[6] LogiCORE IP ChipScope Pro Integrated Logic Analyzer

[7] LogiCORE IP Integrated Logic Analyzer v3.0