Skip to content

Keras, PyTorch and ONNX Accelerators with hls4ml

Integrate machine-learning accelerators generated with hls4ml

Open the guide

Platform support: ESP's HLS flows currently target AMD/Xilinx FPGA technology mappings and are not supported by the Intel/Altera DE10-Pro SX flow. Use an RTL or Third-Party accelerator on that board.
Version-sensitive: the ESP integration sections reflect the current released tree, but the hls4ml source-generation example uses a pinned v0.2-era Vivado HLS flow and still needs end-to-end revalidation.

The video was not recorded with the current ESP release. Use the written guide for the updated ESP integration paths, and read its hls4ml version notice before generating accelerator sources.

What you will learn

  • Integrate into ESP an accelerator designed in Keras, PyTorch or ONNX and generated with hls4ml
    • Generate the accelerator with hls4ml
    • Run an ESP interactive script to integrate the accelerator and generate the Linux device driver and test applications
  • Instantiate the new accelerator in an ESP SoC and test the full system with RTL simulation and on FPGA

Keras, PyTorch and ONNX accelerator workflow

What you will need

What you can read

ESP4ML: Platform-Based Design of Systems-on-Chip for Embedded Machine Learning

Davide Giri, Kuan-lin Chiu, Giuseppe Di Guglielmo, Paolo Mantovani, Luca P. Carloni

(Best Paper Nominee) Design, Automation and Test in Europe Conference (DATE), 2020

Paper Slides Video Tutorial

What you can contribute

The ESP team welcomes external contributions and collaborations, including:

  • Accelerator designs for a wide range of application domains
  • Support for more HLS tools
  • Accelerator design flows from domain-specific languages (DSLs)
  • A power estimation flow

See the contributing guidelines.