--- title: Architecture url: 'https://timeloop.csail.mit.edu/v4/input-formats/design/architecture' markdown: 'https://timeloop.csail.mit.edu/v4/input-formats/design/architecture.md' date: '2024-08-15' description: 'To describe an architecture in Timeloop, one must specify the hardware organization, i.e., the topology of interconnected compute and storage units using the architecture: top-level key. Input Structure The hardware organization is described as a hierarchical tree of branch nodes, with components a…' taxonomy: category: - docs --- [](#) [ Overview ](https://timeloop.csail.mit.edu/) [ Timeloop+Accelergy v4 ](https://timeloop.csail.mit.edu/v4) [ Input Formats ](https://timeloop.csail.mit.edu/v4/input-formats) [ Design ](https://timeloop.csail.mit.edu/v4/input-formats/design) Architecture # Architecture To describe an architecture in Timeloop, one must specify the hardware organization, i.e., the topology of interconnected compute and storage units using the `architecture:` top-level key. # Input Structure The hardware organization is described as a hierarchical tree of branch nodes, with components as leaf nodes. Each branch in the tree describes how sub-nodes connect (*e.g.,* !Hierarchical or !Parallel). Each leaf in the tree describes an element in the architectrue (*e.g.* !Component or !Container). Note that all branch names are adjectives and all leaf names are nouns. One simple-to-describe input is a storage hierarchy. To create a three-level storage hierarchy with a backing storage, a local cache, and a compute unit, we can create a top-level `!Hierarchical` branch and place `!Component` leaves beneath. ```yaml - !Hierarchical nodes: - !Component name: backing_storage ... # Backing storage definition - !Component name: local_cache ... # Local cache definition - !Component name: compute ... # Compute definition ``` ## Leaf Nodes Leaf nodes denote the objects within an architecture. Leaf objects are denoted using an exclamation-point-denoted tag, such as `!Container` or `!Component`. The generalized form is the following: ```yaml - !LEAF_NAME attributes: {Dictionary of attributes} spatial: {meshX: , meshY: } constraints: {Dictionary of constraints} ``` The following types of leaves are supported. Notice that ***The name of each leaf is a noun describing the type of leaf***. - `!Component` objects include storage and compute units. - `!Container` objects encompass other objects. They can be used to group objects together for the purpose of applying constraints or attributes to them. - `!Nothing` objects are represent empty space or a lack of node in a location. For example, a `!Nothing` node may be used in a `!Parallel`branch to process data that skips all other nodes in the branch. Leaf objects have the following sub-keys: - `attributes`: A dictionary of attributes for this leaf. - `spatial`: A description of the number of spatial instances for this leaf. meshX is the number of instances in the X dimension, and meshY in the Y dimension. - `constraints`: Constraints describing the mapping options available to this node. ## Branch Nodes Branch nodes denote the organization of nodes in an architecture. Each branch is a collection of nodes that are related to each other in some way. Branches may contain leaf nodes, other branch nodes, or a mix of both. Branch nodes are denoted with an exclamation-point tag such as `!Hierarchical` or `!Parallel`. Branch objects are denoted in the following way: ```yaml - ! nodes: [List of nodes in this branch] ``` The following types of branches are supported. Notice that ***the name of each branch is an adjective describing the relations between the nodes it contains:*** - `!Hierarchical` nodes can move data between nodes that are lower/higher in the hierarchy. Nodes lower in the hierarchy (further from main memory) can use higher-level nodes as backing memory. - `!Parallel` nodes process disjoint sets of data in parallel. Each dataspace traversing the parallel branch must traverse exactly one of the nodes within the branch. Under the `nodes` key, ## Expressing Spatial Fanout Spatial fanout is expressed by adding a `spatial` attribute to a leaf node. Under a hierarchy, this spatial fanout will be applied to all subsequent nodes in the hierarchy. Multiple spatial fanouts in a hierarchy will be multiplied together. If there is a spatial fanout, then spatial mappings may be constrained by placing constraints under the `constraints.spatial` key. Spatial Spatial constraints are applied *between* the spatial elements in a fanout. That is, if there are 12 PEs in the Y dimension and a spatial constraint introduces a loop `spatial-for P in [0..12)`, then PE would handle a different index of the P dimension. ```yaml nodes: # Top-level hierarchy - !Container name: top - !Container # PE name: PE attributes: {Omitted for brevity} spatial: {meshY: 12} constraints: spatial: {factors: [P=12]} # Each spatial PE handles a different index of P - !Component # Storage name: spad class: storage attributes: {Omitted for brevity} - !Component # MAC unit name: mac class: intmac attributes: {Omitted for brevity} ``` Spatial fanouts are not allowed under pipeline or parallel branches. ## Example Structures ### Parallel Buffers Under our main hierarchy, we can place a `!Parallel` key and introduce two local stores. ```yaml - !Hierarchical nodes: - !Container name: backing_store ... # backing_store defnition - !Parallel # local_store_for_dataspace_A and local_store_for_dataspace_B are parallel nodes: - !Component name: local_store_for_dataspace_A ... # definition - !Component name: local_store_for_dataspace_B ... # definition - !Component # Back out of the parallel block, the compute is lower in the hierarchy below the local stores name: compute ... # definition ``` ### Array of tiles In an accelerator, a tile isn't a physical object, but rather a repeated group of elements such as storage and computation. Here, we can consider a tile as a container for a local storage and a compute unit. We instantiate the tile in a `!Hierarchical` block, so each tile implicitly contains all elements below it. ```yaml - !Hierarchical nodes: - !Component name: backing_store ... # backing_store defnition - !Container name: tile ... # tile defnition spatial: {meshX: 8, meshY: 12} constraints: spatial: permutation: ABCD split: 2 factors: [A=8, B=2, C=3, D=4] - !Component # The local cache is in the tile, so there are 96 total caches name: local_cache ... # definition - !Component # The compute is in the tile, so there are 96 total compute elements name: compute ... # definition ``` ### Array of PEs with X and Y constraints Often, we'd like to apply separate constraints to the X and Y dimensions. In this case, we can split our spatial fanout across two containers. This can allow us to do many interesting things. For example, here we'll allow weight reuse in the X dimension but not the Y dimension. ```yaml - !Hierarchical nodes: - !Component name: backing_store ... # backing_store defnition - !Container name: column_of_tiles ... # tile defnition spatial: {meshX: 8} constraints: spatial: permutation: AB split: 2 # There is no Y here, so we put the split after all dimensions so nothing is assigned to Y. factors: [A=1, B=8] - !Container name: tile ... # tile defnition spatial: {meshY: 12} constraints: spatial: permutation: CD split: 0 # There is no X here, so we put the split as early as possible so everything is assigned to X. factors: [C=3, D=4] no_reuse: [Weights] # Only applies to the Y dimension! - !Component # The local cache is in the tile, so there are 96 total caches name: local_cache ... # definition - !Component # The compute is in the tile, so there are 96 total compute elements name: compute ... # definition ``` # Branch Node Definitions Branch nodes have one sub-key: The `nodes` sub-key, which takes a list of sub-nodes. The top-level `architecture` is a special case of a `!Hierarchical` branch node with an additional `version` key. # Leaf Node Definitions ## `!Component` Leaf Nodes Top-Level Keys The following keys are available to define a leaf node: - name: The name of the component. - class: The class of the component. Generally "storage" or "compute". - subclass: The subclass of the component. Used to query an Accelergy compound component or estimator. - attributes: A dictionary of attributes for the component. - spatial: Description of the spatial organization of the component. - constraints: A dictionary of constraints for the component. - sparse\_optimizations: A dictionary of sparse optimizations for the component. ## The `attributes` key Attributes are a dictionary of key-value pairs that describe the component. They will be used to query Accelergy for energy and area estimates. Attributes used depend on the class and subclass of each component. Classes available are `storage` and `compute`. ### `attributes` for `!Component` Leaf Nodes with `class: storage` Required attributes under the `attributes` key: - `depth` (or `memory_depth`, `data_storage_depth`): the number of "rows" in the storage - `width` (or `memory_width`, `data_storage_width`): number of bits per row - `datawidth` `datawidth` (or `word-bits`, `word_width`): number of bits per word Bandwidth and latency attributes under the `attributes` key: - `shared_bandwidth`: bandwidth provisioned for reads and writes, number of words per cycle - `read_bandwidth`: bandwdith provisioned for reads, number of words per cycle - `write_bandwidth`: bandwidth provisioned for fills/updates, number of words per cycle - `network_fill_latency`: number of cycles to fill a network packet - `network_drain_latency`: number of cycles to drain a network packet - `per_dataspace_bandwidth_consumption_scale`: a dictionary of {dataspace: scale} pairs, where scale is the number of bits consumed per bit of data transferred Sparse optimization attributes under the `attributes` key: - `metadata_datawidth` : number of bits per metadata word - `metadata_depth` : number of rows for metadata storage - `metadata_width` : number of bits per row for metadata storage - `concordant_compressed_tile_traversal`: whether the storage unit can traverse compressed tiles - `tile_partition_supported`: whether the storage unit supports tile partitioning - `decompression_supported`: whether the storage unit supports decompression - `compression_supported`: whether the storage unit supports compression Other attributes under the `attributes` key: - `allow_overbooking`: whether overbooking is allowed - `cluster_size`: number of copies of the storage unit (only one copy active) - `min_utilization`: minimum utilization of the storage unit - `multiple_buffering`: specified as a number, e.g., 2 means double buffering - `has_power_gating`: whether the component has power gating, meaning that it will not leak if it is not used in a given mapping. Additionally, users may define any other attributes that are relevant to their component, which will be used to query Accelergy for energy and area estimates. ## `!Container` Leaf Nodes Top-Level Keys Containers may have the following attributes: - `has_power_gating`: whether the container has power gating, meaning that it will not leak if it is not used in a given mapping. If a container has power gating, then all of its children will have shared power gating available at the level of this container. - name: The name of the container. - attributes: A dictionary of attributes for the container. These are propagated automatically to the attributes of the container's children. - spatial: Description of the spatial organization of the container. - constraints: A dictionary of constraints for the container. - sparse\_optimizations: A dictionary of sparse optimizations for the container. ### A Complete Example ```yaml ``` [accessibility](https://accessibility.mit.edu/) [ ](https://timeloop.csail.mit.edu/v4/input-formats/design) [](https://timeloop.csail.mit.edu/v4/input-formats/design/sparse-optimizations) --- ## Navigation - Parent: [Design](https://timeloop.csail.mit.edu/v4/input-formats/design.md) - Next: [Sparse Optimizations](https://timeloop.csail.mit.edu/v4/input-formats/design/sparse-optimizations.md)