Skip to content

HowToBenchmark

Jump to bottom
Marcus Brinkmann edited this page Jul 29, 2016 · 1 revision

How To Benchmark

Required Hardware

  • One eXternal Benchmarking Harness
  • One eXternal Benmarking Device, CWB will fill out details on how to get/build one

eXternal Benchmarking Harness (XBH)

The XBH interfaces between the PC (using Ethernet) and the XBD (target), using either I²C, UART or Ethernet (UDP). The current workhorse of XBX is the XBH mk.I, based on an ATmega644 microcontroller and an Ethernet chip. If you want to build one for yourself, you only need some inexpensive equipment:

  • 1 AVR-NET-IO board, can be purchased at ​Pollin
  • 1 Break-out board for the DB25 expansion connector of the AVR-NET-IO ​at Pollin
  • 1 ATmega644 microcontroller, since the AVR-NET-IO is shipped with an ATmega32 ​'644 at Pollin

If you have pre-owned hardware with a '644 and an Ethernet controller you can probably use that as XBH, but the hardware setup listed above is tried and true and therfore recommended for beginners.

You will need an In-System-Programmer (ISP) to get the XBH software into the '644. If you do not already own a suitable ISP the ​AVR Dragon is a good choice, since it offers ISP and limited JTAG capability at a decent price point. Just look on ebay or other internet shops for it.

Required Software

  • [download:xbx-2.0.1.tar.gz XBX Base System]
  • [download:xbh-atmega-2.0.1.tar.gz XBH Firmware for Atmega XBHs]
  • At least one platform module
    • [download:platform_lm_3s811_evb-2.0.1.tar.gz Platform support for Stellaris LM3S811 Eval Board, requires Stellars drivers]
    • [download:platform-atmega-2.0.1.tar.gz Platform support for Atmel ATmega 644, 1281 and 1284P]
    • [download:platform_linuxbased-2.0.1.tar.gz Platform support for Linux based targets: Artila M501, NSLU2, FritzBox 7170]
  • One or more algopacks
    • [download:algopack_xbx-2.0.1.tar.gz XBX algopack] - Details and Terms of Use
    • [download:algopack_shavite-2.0.1.tar.gz Shavite algopack] - Details and Terms of Use
    • [download:algopack_groestl-2.0.1.tar.gz Groestl algopack] - Details and Terms of Use
    • [download:algopack_blake-2.0.1.tar.gz Blake algopack] - Details and Terms of Use
    • [download:algopack_avr-crypto-lib-2.0.1.tar.gz AVR-Crypto-Lib algopack] - Details and Terms of Use
    • [download:algopack_arm-crypto-lib-2.0.1.tar.gz ARM-Crypto-Lib algopack] - Details and Terms of Use
    • [download:algopack_sphlib-2.0.1.tar.gz SPHLIB algopack] - Details and Terms of Use
    • [download:algopack_bmw-2.0.1.tar.gz BlueMidnightWish algopack] - Details and Terms of Use
    • [download:algopack_luffa-2.0.1.tar.gz Luffa algopack] - Details and Terms of Use
    • [download:algopack_shabal-tiny-2.0.1.tar.gz Shabal algopack] - Details and Terms of Use
    • [download:algopack_hash3-2.0.1.tar.gz Hash3 conference algopack] - Details and Terms of Use

Getting Started

Hardware and Firmware preparation steps will be filled in by CWB

  1. Extract xbx base archive
  2. Change into the newly created folder and extract all other files there
  3. use ./import_algopack <PATH_TO_ALGOPACK> for each extracted algopack
  4. copy the sample settings file to "settings" cp settings.sample settings
  5. edit the settings file and set the "platform" option to the desired value
  6. run ./compile if no supported compilers are found or anything else undesirable happens, the script will notify you
  7. run ./execute <IP_OF_XBH> to start the benchmarking process
  8. after the benchmark is complete, a gz-file called xbxdata-HOSTNAME-IP_OF_XBH-TIMESTAMP.gz will be created
  9. the non-compressed results can be found in bench/PLATFORM_NAME-IP_OF_XBH/ The tools folder contains some helpful scripts for viewing results or errors:

All helper scripts reside in the subdirectory tools and expect their working directory to be the directory where compile and execute reside. In layman's terms: execute all helper scripts by typing tools/<toolname>

Commands Useful After a compile-Run

Command Explanation
build_stats Outputs a simple table containing the build results
compilererrors Lists the output of by the compilers
binaries_wiki_table Outputs a wiki style table with information about binaries
Commands Useful After an execute-Run
Command Explanation
try_wiki_table Outputs a wiki style table containing the results of the checksum tests and quick benchmarks
measure_wiki_table Outputs a wiki style table containing the detailed measurement results
drift_wiki_table Outputs a wiki style table containing the output of the drift measurements done during execution
stack_wiki_table Outputs a wiki style table containing stack measurements
try_cipher_plot Draws a plot using gnuplot containing the different implementation speeds of a certain cipher

General Tools

Command Explanation
tocsv Converts wiki style tables (STDIN) to csv (STDOUT)

Questions / suggestions / new platforms or new implementations? Mail to ​[email protected]

Clone this wiki locally