Architecture

To describe an architecture in Timeloop, one must specify the hardware organization, i.e., the topology of interconnected compute and storage units using the architecture: top-level key.

Input Structure

The hardware organization is described as a hierarchical tree of branch nodes, with components as leaf nodes. Each branch in the tree describes how sub-nodes connect (e.g., !Hierarchical or !Parallel). Each leaf in the tree describes an element in the architectrue (e.g. !Component or !Container). Note that all branch names are adjectives and all leaf names are nouns.

One simple-to-describe input is a storage hierarchy. To create a three-level storage hierarchy with a backing storage, a local cache, and a compute unit, we can create a top-level !Hierarchical branch and place !Component leaves beneath.

- !Hierarchical
  nodes:
  - !Component
    name: backing_storage
    ... # Backing storage definition

  - !Component
    name: local_cache
    ... # Local cache definition

  - !Component
    name: compute
    ... # Compute definition

Leaf Nodes

Leaf nodes denote the objects within an architecture.

Leaf objects are denoted using an exclamation-point-denoted tag, such as !Container or !Component. The generalized form is the following:

- !LEAF_NAME
  attributes: {Dictionary of attributes}
  spatial: {meshX: <int>, meshY: <int>}
  constraints: {Dictionary of constraints}

The following types of leaves are supported. Notice that The name of each leaf is a noun describing the type of leaf.

• !Component objects include storage and compute units.
• !Container objects encompass other objects. They can be used to group objects together for the purpose of applying constraints or attributes to them.

• !Nothing objects are represent empty space or a lack of node in a location. For example, a !Nothing node may be used in a !Parallel branch to process data that skips all other nodes in the branch.

  Leaf objects have the following sub-keys:

  • attributes: A dictionary of attributes for this leaf.
  • spatial: A description of the number of spatial instances for this leaf. meshX is the number of instances in the X dimension, and meshY in the Y dimension.
  • constraints: Constraints describing the mapping options available to this node.

Branch Nodes

Branch nodes denote the organization of nodes in an architecture. Each branch is a collection of nodes that are related to each other in some way. Branches may contain leaf nodes, other branch nodes, or a mix of both. Branch nodes are denoted with an exclamation-point tag such as !Hierarchical or !Parallel.

Branch objects are denoted in the following way:

- !<BRANCHTYPE_ADJECTIVE>
  nodes: [List of nodes in this branch]

The following types of branches are supported. Notice that the name of each branch is an adjective describing the relations between the nodes it contains:

• !Hierarchical nodes can move data between nodes that are lower/higher in the hierarchy. Nodes lower in the hierarchy (further from main memory) can use higher-level nodes as backing memory.

• !Parallel nodes process disjoint sets of data in parallel. Each dataspace traversing the parallel branch must traverse exactly one of the nodes within the branch.

  Under the nodes key,

Expressing Spatial Fanout

Spatial fanout is expressed by adding a spatial attribute to a leaf node. Under a hierarchy, this spatial fanout will be applied to all subsequent nodes in the hierarchy. Multiple spatial fanouts in a hierarchy will be multiplied together.

If there is a spatial fanout, then spatial mappings may be constrained by placing constraints under the constraints.spatial key. Spatial Spatial constraints are applied between the spatial elements in a fanout. That is, if there are 12 PEs in the Y dimension and a spatial constraint introduces a loop spatial-for P in [0..12), then PE would handle a different index of the P dimension.

nodes: # Top-level hierarchy
- !Container
  name: top

- !Container # PE
  name: PE
  attributes: {Omitted for brevity}
  spatial: {meshY: 12}
  constraints:
    spatial: {factors: [P=12]} # Each spatial PE handles a different index of P

- !Component # Storage
  name: spad
  class: storage
  attributes: {Omitted for brevity}

- !Component # MAC unit
  name: mac
  class: intmac
  attributes: {Omitted for brevity}

Spatial fanouts are not allowed under pipeline or parallel branches.

Example Structures

Parallel Buffers

Under our main hierarchy, we can place a !Parallel key and introduce two local stores.

- !Hierarchical
  nodes:
  - !Container
    name: backing_store
    ... # backing_store defnition

  - !Parallel # local_store_for_dataspace_A and local_store_for_dataspace_B are parallel
    nodes:
    - !Component
      name: local_store_for_dataspace_A
      ... # definition
    - !Component
      name: local_store_for_dataspace_B
      ... # definition

  - !Component # Back out of the parallel block, the compute is lower in the hierarchy below the local stores
    name: compute
    ... # definition

Array of tiles

In an accelerator, a tile isn't a physical object, but rather a repeated group of elements such as storage and computation. Here, we can consider a tile as a container for a local storage and a compute unit.

We instantiate the tile in a !Hierarchical block, so each tile implicitly contains all elements below it.

- !Hierarchical
  nodes:
  - !Component
    name: backing_store
    ... # backing_store defnition

  - !Container
    name: tile
    ... # tile defnition
    spatial: {meshX: 8, meshY: 12}
    constraints:
        spatial:
            permutation: ABCD
            split: 2
            factors: [A=8, B=2, C=3, D=4]

  - !Component # The local cache is in the tile, so there are 96 total caches
    name: local_cache
    ... # definition

  - !Component # The compute is in the tile, so there are 96 total compute elements
    name: compute
    ... # definition

Array of PEs with X and Y constraints

Often, we'd like to apply separate constraints to the X and Y dimensions. In this case, we can split our spatial fanout across two containers. This can allow us to do many interesting things. For example, here we'll allow weight reuse in the X dimension but not the Y dimension.

- !Hierarchical
  nodes:
  - !Component
    name: backing_store
    ... # backing_store defnition

  - !Container
    name: column_of_tiles
    ... # tile defnition
    spatial: {meshX: 8}
    constraints:
        spatial:
            permutation: AB
            split: 2 # There is no Y here, so we put the split after all dimensions so nothing is assigned to Y.
            factors: [A=1, B=8]

  - !Container
    name: tile
    ... # tile defnition
    spatial: {meshY: 12}
    constraints:
        spatial:
            permutation: CD
            split: 0 # There is no X here, so we put the split as early as possible so everything is assigned to X.
            factors: [C=3, D=4]
            no_reuse: [Weights] # Only applies to the Y dimension!

  - !Component # The local cache is in the tile, so there are 96 total caches
    name: local_cache
    ... # definition

  - !Component # The compute is in the tile, so there are 96 total compute elements
    name: compute
    ... # definition

Branch Node Definitions

Branch nodes have one sub-key: The nodes sub-key, which takes a list of sub-nodes.

The top-level architecture is a special case of a !Hierarchical branch node with an additional version key.

Leaf Node Definitions

!Component Leaf Nodes Top-Level Keys

The following keys are available to define a leaf node:

• name: The name of the component.
• class: The class of the component. Generally "storage" or "compute".
• subclass: The subclass of the component. Used to query an Accelergy compound component or estimator.
• attributes: A dictionary of attributes for the component.
• spatial: Description of the spatial organization of the component.
• constraints: A dictionary of constraints for the component.
• sparse_optimizations: A dictionary of sparse optimizations for the component.

The attributes key

Attributes are a dictionary of key-value pairs that describe the component. They will be used to query Accelergy for energy and area estimates. Attributes used depend on the class and subclass of each component.

Classes available are storage and compute.

attributes for !Component Leaf Nodes with class: storage

Required attributes under the attributes key:

• depth <REQUIRED> (or memory_depth, data_storage_depth): the number of "rows" in the storage
• width<REQUIRED> (or memory_width, data_storage_width): number of bits per row
• datawidth <REQUIRED> datawidth (or word-bits, word_width): number of bits per word

Bandwidth and latency attributes under the attributes key:

• shared_bandwidth: bandwidth provisioned for reads and writes, number of words per cycle
• read_bandwidth: bandwdith provisioned for reads, number of words per cycle
• write_bandwidth: bandwidth provisioned for fills/updates, number of words per cycle
• network_fill_latency: number of cycles to fill a network packet
• network_drain_latency: number of cycles to drain a network packet
• per_dataspace_bandwidth_consumption_scale: a dictionary of {dataspace: scale} pairs, where scale is the number of bits consumed per bit of data transferred

Sparse optimization attributes under the attributes key:

• metadata_datawidth <REQUIRED IF METADATA IS USED> : number of bits per metadata word
• metadata_depth <REQUIRED IF METADATA IS USED> : number of rows for metadata storage
• metadata_width <REQUIRED IF METADATA IS USED>: number of bits per row for metadata storage
• concordant_compressed_tile_traversal: whether the storage unit can traverse compressed tiles
• tile_partition_supported: whether the storage unit supports tile partitioning
• decompression_supported: whether the storage unit supports decompression
• compression_supported: whether the storage unit supports compression

Other attributes under the attributes key:

• allow_overbooking: whether overbooking is allowed
• cluster_size: number of copies of the storage unit (only one copy active)
• min_utilization: minimum utilization of the storage unit
• multiple_buffering: specified as a number, e.g., 2 means double buffering
• has_power_gating: whether the component has power gating, meaning that it will not leak if it is not used in a given mapping.

Additionally, users may define any other attributes that are relevant to their component, which will be used to query Accelergy for energy and area estimates.

!Container Leaf Nodes Top-Level Keys

Containers may have the following attributes:

• has_power_gating: whether the container has power gating, meaning that it will not leak if it is not used in a given mapping. If a container has power gating, then all of its children will have shared power gating available at the level of this container.
• name: The name of the container.
• attributes: A dictionary of attributes for the container. These are propagated automatically to the attributes of the container's children.
• spatial: Description of the spatial organization of the container.
• constraints: A dictionary of constraints for the container.
• sparse_optimizations: A dictionary of sparse optimizations for the container.

A Complete Example