SST-HGCC
SST-HGCC (HGCC) is Mercury's compile-time toolchain for running HPC applications
inside the SST discrete-event simulator. It
provides hgcc and hg++ compiler wrappers plus an LLVM-based source rewriter
(ssthg_clang) that transform user C/C++ code into a loadable shared library
(.so) executed by the Mercury element in sst-elements.
HGCC is not a standalone MPI runtime. You need SST Core, sst-elements (with the Mercury/HG element), and LLVM/Clang with libTooling to build and use it.
For step-by-step install commands, see INSTALL.md.
How it works
At build time, hg++ preprocesses your source, runs ssthg_clang to
skeletonize compute and rewrite main, then compiles and links a PIC shared
object. At simulation time, an SST Python script configures the network
(Merlin) and operating system (Mercury), which loads your .so and runs the
rewritten entry point.
How code runs under Mercury
When the simulator loads your .so, code is not running natively the
way it would under mpirun. A few things change:
mainis renamed.ssthg_clangrenamesmaintosst_hg_user_main_<mangled>and emits a wrapper Mercury calls once per rank inside a single virtual OS process. There is nompiexec; ranks are virtual.- MPI is virtual.
#include <mask_mpi.h>routes every MPI call through sst-elements'mask_mpi, which models messages on the simulated Merlin network. Real MPI is never linked or invoked. - Compute is modeled, not executed.
ssthg_clangstrips loops and expressions by default (skeletonize mode), insertingssthg_*time-advance hooks driven by the simulated platform (frequency,compute_library_access_width,compute_library_loop_overhead). Values are no longer meaningful; time is. Use#pragma sst keepto preserve real computation where you need it. - Globals and TLS are privatized. All ranks residing in an sst-core simulation process share one address space, so
the rewriter rewrites globals and thread-locals to per-rank storage. The
tests/test_tls.ccintegration test demonstrates this — every rank printsmy_global: 1because each++my_globalhits a private copy. - Time advances via platform params. Wall-clock latency comes from the
Merlin/Mercury platform definition (link latency/bandwidth,
post_rdma_delay,max_eager_msg_size, etc.), not from the host CPU.
Getting Started
Prerequisites
Install these in order before building sst-hgcc:
| Component | Notes |
|---|---|
| Autotools | autoconf, automake, libtool (used by ./autogen.sh) |
| LLVM 22 | With libTooling (required for ssthg_clang) |
| C/C++ compiler | Clang recommended (CC=clang CXX=clang++) |
| SST Core | Provides sst-config on your PATH |
| sst-elements | Mercury/HG element; build with --with-std=17 |
Build and install
git clone https://github.com/sstsimulator/sst-hgcc.git
cd sst-hgcc
./autogen.sh
mkdir build && cd build
../configure CC=clang CXX=clang++ \
--with-std=17 \
--prefix=$PREFIX \
--with-sst-core=$PREFIX \
--with-sst-elements=$PREFIX \
--with-clang=$PREFIX
make -j$(nproc) && make install
export PATH=$PREFIX/bin:$PATH
On macOS, also set:
export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
export LDFLAGS="-fuse-ld=lld"
Verify the install
hg++ --version
hg++ --flags # print Mercury include/link flags added automatically
make check # lit rewriter tests + test_tls library (optional)
make installcheck # SST integration test via tests/test_tls (needs sst in PATH)
The examples under examples/ are not built by make, make install, or
make check. See Building the examples below.
Configure-time options
| Option | Description |
|---|---|
--prefix=DIR | Install root (default /usr/local) |
--with-sst-core=DIR | Required. SST Core install prefix |
--with-sst-elements=DIR | Required. sst-elements install prefix |
--with-clang=DIR | Required for rewriter. LLVM install root |
--with-std={11,14,17} | C++ standard for user apps (default 11; 17 recommended) |
--enable-use-replacements | Install extra replacement STL headers |
--enable-strict-tests | Fail configure if lit, FileCheck, or sst missing |
--with-python=DIR | Python install prefix |
--with-sdk=PATH | macOS SDK path (-isysroot) |
hgcc / hg++ command-line flags
| Flag | Description |
|---|---|
--skeletonize | Activate skeletonization (strip compute, rewrite main; default for app builds) |
--memoize | Activate memoization mode (capture variable types) |
--replacements=hdr1,hdr2 | Inject replacement headers (e.g. pthread.h,vector) |
--disable-mpi | Skip virtual MPI environment |
--app-name=NAME | Override app registration name (default: derived from -o output) |
--sst-component | Skip all source-to-source rewriting (use when building an SST component itself) |
--flags | Print extra flags SST adds automatically |
--prefix | Print the hgcc install prefix |
--host-cc, --host-cxx | Override underlying host compilers |
-o, -c, -std, -I, -L, -l, -fPIC | Standard compile/link flags (forwarded) |
Environment variables
| Variable | Description |
|---|---|
SST_HG_VERBOSE=1 | Verbose compiler wrapper output |
SST_HG_DELETE_TEMPS=0 | Keep temporary source-to-source files |
SST_HG_DELETE_TEMP_SOURCES=0 | Keep rewritten .cc intermediates |
SST_HG_DELETE_TEMP_OBJECTS=0 | Keep temporary object files |
SST_HG_SKELETONIZE | Enable skeletonization via environment |
SST_HG_MEMOIZE | Enable memoization via environment |
SST_HG_DEBUG_SRC2SRC | Debug the source-to-source pipeline |
SST_HG_CXX, SST_HG_CC | Override host C++ / C compilers |
SST_HG_PREFIX | Override hgcc install prefix |
SST_HG_CONFIG=1 | Skip certain steps (used by automake/cmake probes) |
Install layout
After make install, key paths under $prefix are:
| Path | Contents |
|---|---|
$prefix/bin/ | hgcc, hg++, ssthg_clang, Python driver scripts |
$prefix/include/replacements/ | Shadow STL/system/MPI/pthread headers |
$prefix/include/memoization/ | Memoization capture headers (C++17) |
$prefix/include/hgcc/ | hgcc-specific headers |
Building the examples
The examples/ directory contains documented Mercury apps and
#pragma sst demos. These are optional and kept separate from the main
toolchain build so that a normal install stays fast and does not require a full
Mercury/SST stack at compile time.
What each make target builds
| Target | Builds examples? | Notes |
|---|---|---|
make | No | Builds hgcc, hg++, ssthg_clang, and wrapper scripts only |
make install | No | Installs the toolchain; does not install example .so files |
make check | No | Runs lit rewriter tests; builds tests/libtest_tls.so for integration testing |
make examples | Yes | Compiles all example sources (see below) |
make install-examples | Installs only | Copies the four runnable example .so files to the SST element library path |
make run-examples | Yes (if needed) | Runs make examples, then executes the four SST-runnable demos |