Guide – How to: setup

Latest update: 2023-01-23

This guide illustrates prerequisites and environment setup for ESP. For a quicker and more portable setup, we also provide Docker images for ESP as described in the Docker guide.

Table of Contents

Software packets
CAD tools
Environment variables
ESP repository
Software toolchain
Patching Ariane for Xcelium Simulator

Software packets

The ESP design flow has been developed and tested on the CentOS 7 Linux distribution, which is our preferred OS. Alternatively, you can use Ubuntu 18.04 or Red Hat Enterprise Linux 7.8.

Note: Cadence tools do not officially support Ubuntu 18.04. If you choose to work with Ubuntu, the accelerators unit test simulation within the Cadence Stratus HLS environment may not function properly.

In order to support embedded scripts and CAD tools for simulation and implementation, the following packages should be installed using the system packet manager.

Centos 7

  # Miscellaneous
  sudo yum install -y epel-release
  sudo yum install -y git octave octave-io jq

  # Python
  sudo yum install -y python python-pip python3 python3-pip python3-tkinter
  sudo pip3 install Pmw

  # Perl
  sudo yum install -y perl perl-Env perl-YAML perl-XML-Simple
  sudo yum install -y perl-ExtUtils-MakeMaker perl-Thread-Queue perl-Capture-Tiny

  # CAD tools and SW toolchains dependencies
  sudo yum install -y xterm
  sudo yum install -y csh ksh zsh tcl
  sudo yum install -y glibc-devel glibc-devel.i686
  sudo yum install -y glibc-static glibc-static.i686
  sudo yum install -y mesa-libGL.i686 mesa-libGLU.i686
  sudo yum install -y mesa-libGL mesa-libGLU
  sudo yum install -y mesa-dri-drivers mesa-dri-drivers.i686
  sudo yum install -y readline-devel readline-devel.i686
  sudo yum install -y libXp libXp.i686
  sudo yum install -y openmotif
  sudo yum install -y ncurses
  sudo yum install -y gdbm-devel gdbm-devel.i686
  sudo yum install -y libSM libSM.i686
  sudo yum install -y libXcursor libXcursor.i686
  sudo yum install -y libXft libXft.i686
  sudo yum install -y libXrandr libXrandr.i686
  sudo yum install -y libXScrnSaver libXScrnSaver.i686
  sudo yum install -y libmpc-devel libmpc-devel.i686
  sudo yum install -y nspr nspr.i686
  sudo yum install -y nspr-devel nspr-devel.i686
  sudo yum install -y tk tk-devel
  sudo yum install -y dtc bison flex bzip2 patch bc
  sudo yum install -y Xvfb
  sudo ln -s /lib64/libtiff.so.5 /lib64/libtiff.so.3
  sudo ln -s /usr/lib64/libmpc.so.3 /usr/lib64/libmpc.so.2
  sudo yum install boost boost-thread boost-devel

  # For older GUIs (e.g. Stratus)
  sudo yum install -y libpng12 libpng12.i686

  # QT
  sudo yum install -y qtcreator
  sudo ln -s /usr/bin/qmake-qt5 /usr/bin/qmake

Ubuntu 18.04

  # Miscellaneous
  sudo apt install -y git octave octave-io jq

  # Python
  sudo apt install -y python python-pip python3 python3-pip python3-tk
  sudo pip3 install Pmw # sudo may not be needed

  # Perl
  sudo apt install -y perl libyaml-perl libxml-perl

  # CAD tools and SW toolchains dependencies
  sudo apt install -y xterm
  sudo apt install -y csh ksh zsh tcl
  sudo apt install -y build-essential
  sudo apt install -y libgl1-mesa-dev libglu1-mesa libgl1-mesa-dri
  sudo apt install -y libreadline-dev
  sudo apt install -y libxpm-dev
  sudo apt install -y libmotif-dev
  sudo apt install -y libncurses5
  sudo apt install -y libncurses-dev
  sudo apt install -y libgdbm-dev
  sudo apt install -y libsm-dev
  sudo apt install -y libxcursor-dev
  sudo apt install -y libxft-dev
  sudo apt install -y libxrandr-dev
  sudo apt install -y libxss-dev
  sudo apt install -y libmpc-dev
  sudo apt install -y libnspr4
  sudo apt install -y libnspr4-dev
  sudo apt install -y libboost-all-dev
  sudo apt install -y tk tk-dev
  sudo apt install -y flex
  sudo apt install -y rename
  sudo apt install -y zlib1g:i386
  sudo apt install -y gcc-multilib
  sudo apt install -y device-tree-compiler
  sudo apt install -y bison
  sudo apt install -y xvfb

  # For older GUIs (e.g. Stratus)
  echo 'deb http://security.ubuntu.com/ubuntu xenial-security main' | sudo tee -a /etc/apt/sources.list
  sudo apt install -y libpng12-0

  # QT
  sudo apt install -y qtcreator

Red Hat Enterprise Linux 7.8

  # Miscellaneous
  sudo yum install -y epel-release
  sudo yum install -y git octave octave-io jq

  # Python
  sudo yum install -y python python-pip python3 python3-pip python34-tkinter
  sudo pip3 install Pmw

  # Perl
  sudo yum install -y perl perl-YAML perl-ExtUtils-MakeMaker
  sudo yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/perl-XML-Simple-2.20-5.el7.noarch.rpm

  # CAD tools and SW toolchains dependencies
  sudo yum install -y xterm
  sudo yum install -y csh ksh zsh tcl
  sudo yum install -y glibc-devel glibc-devel.i686
  sudo yum install -y glibc-static glibc-static.i686
  sudo yum install -y mesa-libGL.i686 mesa-libGLU.i686
  sudo yum install -y mesa-libGL mesa-libGLU
  sudo yum install -y mesa-dri-drivers mesa-dri-drivers.i686
  sudo yum install -y readline-devel readline-devel.i686
  sudo yum install -y libXp libXp.i686
  sudo yum install -y openmotif
  sudo yum install -y ncurses
  sudo yum install -y gdbm-devel gdbm-devel.i686
  sudo yum install -y libSM libSM.i686
  sudo yum install -y libXcursor libXcursor.i686
  sudo yum install -y libXft libXft.i686
  sudo yum install -y libXrandr libXrandr.i686
  sudo yum install -y libXScrnSaver libXScrnSaver.i686
  sudo yum install -y libmpc-devel
  sudo yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/libmpc-1.0.1-3.el7.i686.rpm
  sudo yum install -y nspr nspr.i686
  sudo yum install -y nspr-devel nspr-devel.i686
  sudo yum install -y tk tk-devel
  sudo yum install -y Xvfb
  sudo yum install -y http://mirror.centos.org/centos/7/extras/x86_64/Packages/dtc-1.4.6-1.el7.x86_64.rpm
  sudo ln -s /lib64/libtiff.so.5 /lib64/libtiff.so.3
  sudo ln -s /usr/lib64/libmpc.so.3 /usr/lib64/libmpc.so.2
  sudo yum install -y bison

  # For older GUIs (e.g. Stratus)
  sudo yum install -y libpng12 libpng12.i686

  # QT
  sudo yum install -y qtcreator
  sudo ln -s /usr/bin/qmake-qt5 /usr/bin/qmake

CAD tools

ESP leverages a mix of open-source tools and scripts, as well as commercial tools. The following list specifies which commercial tools are currently supported and which tools are required to complete some steps of the ESP design methodology. Support for the tools marked as beta in the following list has not been fully tested or completed.

Required

RTL simulation: requires one of the following tools (64-bit version)
- Mentor Graphics ModelSim SE 2019.2: RTL system-level simulator
- Cadence Incisive 15.2: RTL system-level simulator (LEON3 only)
- Cadence Xcelium 19.03: RTL system-level simulator (apply patch below)
FPGA prototyping: requires the following tool
- Xilinx Vivado 2019.2: logic synthesis and implementation for FPGA

Optional

Accelerator design: the following tools are optional
- Cadence Stratus HLS 20.24: high-level synthesis of accelerators from SystemC
- Xilinx Vivado HLS 2019.2: high-level synthesis of accelerators from C/C++ (beta)
- Mentor Catapult HLS 10.5b: high-level synthesis of accelerators from SystemC and C/C++
- Xilinx Vivado 2018.3: requirement of Catapult HLS
Cache hierarchy: the default ESP configuration selects an RTL implementation of the cache hierarchy (no additional tool required). This supports both multi-core execution and coherence for accelerators. Optionally, ESP provides a SystemC implementation of the caches that can be synthesized with Cadence Stratus HLS. The SystemC version of the caches is particularly helpful to conduct architectural research on coherence, as the SystemC model is significantly easier to modify than the RTL implementation.

Environment variables

Here are the environment variables required by the CAD tools:

# Cadence: Stratus HLS, Incisive, Xcelium
  # e.g. <stratus_path> = /opt/cadence/stratus2024
  # e.g. <incisive_path> = /opt/cadence/incisive152
  # e.g. <xcelium_path> = /opt/cadence/xcelium1903
  export LM_LICENSE_FILE=$LM_LICENSE_FILE:<cadence_license_path>
  export PATH=$PATH:<stratus_path>/bin:
  # De-comment either Incisive or Xcelium paths, but not both!
  # export PATH=$PATH:<incisive_path>/tools/cdsgcc/gcc/bin
  # export PATH=$PATH:<incisive_path>/tools/bin
  export PATH=$PATH:<xcelium_path>/tools/cdsgcc/gcc/bin
  export PATH=$PATH:<xcelium_path>/tools/bin
  export CDS_AUTO_64BIT=all
  export HOST=$(hostname) # for Ubuntu only

# Xilinx: Vivado, Vivado HLS
  # e.g. <vivado_path> = /opt/xilinx/Vivado/2019.2
  export XILINXD_LICENSE_FILE=<xilinx_license_path>
  source <vivado_path>/settings64.sh

# Mentor: Catapult HLS, Modelsim
  # e.g. <modelsim_path> = /opt/mentor/modeltech
  # e.g. <catapult_path> = /opt/mentor/catapult
  export LM_LICENSE_FILE=$LM_LICENSE_FILE:<mentor_license_path>
  export PATH=$PATH:<modelsim_path>/bin
  export AMS_MODEL_TECH=<modelsim_path>
  export PATH=$PATH:<catapult_path>/Mgc_home/bin
  export SYSTEMC=<catapult_path>/Mgc_home/shared
  export CATAPULT_HOME=<catapult_path>/Mgc_home
  export LIBDIR=-L<catapult_path>/Mgc_home/shared/lib $LIBDIR

# RISC-V (for Ariane and Ibex)
  # e.g. <riscv_path> = /opt/riscv
  # e.g. <riscv32imc_path> = /opt/riscv32imc
  export RISCV=<riscv_path>
  export RISCV32IMC=<riscv32imc_path>
  export PATH=$PATH:<riscv_path>/bin:<riscv32imc_path>/bin

# Leon3
  # e.g. <leon3_path> = /opt/leon
  export PATH=$PATH:<leon3_path>/bin
  export PATH=$PATH:<leon3_path>/mklinuximg
  export PATH=$PATH:<leon3_path>/sparc-elf/bin

ESP repository

ESP is available as an open-source GitHub repository. All original files in ESP are governed by the Apache 2.0 License, whereas third-party open-source software maintains its original license and includes the copyright notice from the authors.

Use the following command to obtain the ESP source code from GitHub. The option recursive will initialize and download the source code from submodules that are linked by the main ESP repository. Please be patient, as one of the submodules is Linux, which embeds a very large history.

# Clone ESP and all Git submodules at once
git clone --recursive https://github.com/sld-columbia/esp.git

# Alternatively
git clone https://github.com/sld-columbia/esp.git
git submodule update --init --recursive

Software toolchain

Every ESP SoC must include at least one processor core to execute the operating system and the target applications. ESP currently supports the Leon3 core from GRLIB, implementing the 32-bit SPARC V8 instruction-set architecture (ISA) and the Ariane core from ETH Zurich, implementing the 64-bit version of the RISC-V ISA.

Since the target ISA is not x86, a cross compiler is required to build software that can execute on the target processor. In addition, Linux requires a root filesystem that hosts all necessary initialization scripts, header files and dynamically-linked libraries. A partial overlay for the root filesystem is embedded into the ESP code base, however, binaries from busybox and libraries must be compiled and combined to the overlay. Failure to complete this step will prevent Linux from completing the boot process.

ESP provides scripts that allow users to build the toolchain automatically.

cd <esp>

# Leon3 toolchain
./utils/toolchain/build_leon3_toolchain.sh

# RISC-V 64-bit toolchain (for the Ariane processor)
./utils/toolchain/build_riscv_toolchain.sh

# RISC-V 32-bit toolchain (for the Ibex processor)
./utils/toolchain/build_riscv32imc_toolchain.sh

The scripts go through several interactive steps to build the toolchain. If one step fails due to missing packages on the host system, users can restart the script and skip phases that completed successfully. For a default installation the user can select all the default answers of the interactive script.

Patching Ariane for Xcelium Simulator

The source code of Ariane includes UVM and SVA features that can only work with Modelsim. These features are disabled by the ESP Makefile by defining the variable VERILATOR during compilation. As a result, the instruction tracer is not instantiated, because it requires UVM support. The developers of Ariane replaced the instruction tracer with custom RTL code when simulating with Verilator. This code, unfortunately, triggers non suppressible errors in the Xcelium compiler.

While we do recommend to use Modelsim for simulation and keep all of the UVM and SVA features enabled, if you need to simulate using Xcelium, please apply this patch to the Ariane repository in rtl/cores/ariane/ariane. The patch is simply commenting the code for instruction tracing with Verilator.

cd rtl/cores/ariane/ariane
cp path/to/xcelium_ariane.patch .
git apply xcelium_ariane.patch

Finally, please note that Incisive cannot compile the source code of Ariane, but it will work with Leon3-based ESP instances.