Skip to main content

Shared Objects

Shared Objects are special read-only data structures that can be shared across Components. They are intended as a memory use optimization when many Components would otherwise replicate read-only data (e.g., large routing tables). In parallel simulations, the object is replicated once on each SST rank and is shared among threads. Shared objects are writeable through SST's construction and init stages and become read-only once the setup stage begins.

The available objects are:

  • SharedArray<T>
  • SharedSet<T>
  • SharedMap<K,V>

The templated types (T,K,V) must be non-pointer types and must be serializable.

Obtaining a shared object​

Shared objects must be created and initialized during SST's construction and/or init stages. Objects are writeable during this time and use a well-defined policy to resolve write conflicts. To obtain an object, call object.initialize(NAME) with the name of the object. A conflict resolution policy different from the default can also be specified.

Initializing a shared object named 'shared_set'
SST::SharedSet<int>   set;
set.initialize("shared_set");

Writing a shared object​

After initializing, components can write the object using write functions provided by the shared object class. These functions check for write conflicts and ensure atomic updates when components are writing simultaneously. Thread-safe read functions are also provided. Components may write during SST's construction and init stages. Once a component finishes writing, it should call object.publish() to indicate it is done. A component may not write the object after publishing it, although other components may continue to write until they call publish(). If a component fails to publish the object, it will be automatically published at the start of the setup stage. Writes from components on different ranks are merged and made globally available at the end of each init phase.

Resolving write conflicts​

Write conflicts occur when components write to the same element in the shared object. There are three conflict resolution policies: FE_VERIFY, INIT_VERIFY, and NO_VERIFY. Different object types have different defaults.

FE_VERIFY​

This policy tracks whether an element has already been written or not. An error is flagged if multiple writes with different values occur to the same element. Default for: SharedMap, SharedSet.

INIT_VERIFY​

For SharedArray which can be initialized to a non-default value, this policy tracks whether an element is uninitialized, initialized, or written by a component. An error occurs if multiple writes with different values occur but overwriting the initialized value is allowed. If the element was not given an initial value, this policy is identical to FE_VERIFY. Default for: SharedArray.

NO_VERIFY​

Under this policy, elements are not checked for duplicate writes. No ordering guarantee is provided so this policy is useful when components are guaranteed to not write the same element. Default for: None.

Accessing shared objects during simulation​

During simulation, components may read shared objects using mutex-free accessors.

Cleaning up shared objects​

No clean up is needed, shared objects are managed by SST-Core.

Example: Using a SharedArray to randomize physical address maps​

The following example comes from the miranda element library and is an example of one (or a few) components populating a shared object for consumption by all components of the same type. Miranda simulates a processor core running a workload generated by a synthetic trace. Each miranda core is a separate component and during construction, the cores generate a lookup table to control memory page mapping. The page table uses a SharedArray to avoid having to replicate the same large table at each component. The table is generated using a deterministic mapping defined in the miranda parameters. At least one miranda component populates the table and the remaining ones read it to perform virtual to physical page translations during simulation.

class MirandaMemoryManager {
public:
    MirandaMemoryManager(...)
    {
        // Initialize the pageMap to have one entry for each page in memory (pageCount)
        int order = pageMap.initialize("miranda_pagemap", pageCount);

        /* If order == 0, this is the first component on its rank to initialize this shared array.
         * As an optimization, to avoid mutex contention, if another miranda component has already 
         * initialized the array, then this miranda component will not initialize it.
         * If miranda components happen to be placed on different ranks, multiple components will 
         * populate the array. However, they generate identical mappings so no write conflict arises.
         */
        if (order == 0) { 
            // Generate a local array of the mappings
            uint64_t * pageArr = (uint64_t*) malloc(pageCount * sizeof(uint64_t));
            for (uint64_t i = 0; i < pageCount; ++i>) {
                pageArr[i] = ... /* See sst-elements/src/sst/elements/miranda/mirandaMemMgr.h for full code */
            }

            for (int i = 0; i < pageCount; ++i) {
                pageMap.write(i, pageArr[i]);
            }
        }
    }

    uint64_t mapAddress(const uint64_t addrIn) const {
        // Compute virtual page index
        const uint64_t pageOffset = addrIn % pageSize;
        const uint64_t virtualPageStart = (addrIn - pageOffset) / pageSize;

        // Lookup physical address for the given virtual page using the SharedArray
        const uint64_t physAddress = pageMap[virtualPageStart] + pageOffset;

        return physAddress;
    }

private:
    uint64_t pageSize;
    uint64_t pageCount;
    uint64_t maxMemoryAddress;
    SST::Output* output;
    Shared::SharedArray<uint64_t> pageMap;
};

#include <sst/core/sharedArray.h>
// OR
#include <sst/core/sharedMap.h>
// OR
#include <sst/core/sharedSet.h>