What Is Graphic Cache Size
Processing power has increased at a faster rate than memory access speed throughout the history of computers, and as this gap, and thus the cost of memory access, grew, it became necessary to introduce intermediate high-speed storage resources between the processor and memory to be able to feed the former with data at a sufficient rate, and thus processor caches were born.
Learn graphic design online through Blue Sky Graphics online graphic design course in the UK to get acquainted with this field on an industrial level.
Fundamentals of Graphic Cache
Caches’ fundamental utility is that they offer buffering, and in this regard, caches and buffers perform comparable functions:
Reduce latency by reading data from memory in bigger chunks, with the expectation that following data requests would target neighbouring places.
Increase performance by grouping together several tiny transfers to create bigger, more efficient memory requests.
Intel Core 2 Duo cache structure.
CPUs nowadays have more complex cache hierarchies, as they typically have per-core L2 caches and an additional L3 cache shared across the cores (sometimes even a L4 cache as well), but we chose the Core 2 Duo because its cache hierarchy captures the essence of multi-core processor cache organisation. Multi-core CPUs, in particular, usually have at least two layers of cache:
The CPU core’s nearest level cache is private.
All CPU cores share the last level cache.
SRAM versus DRAM: Cache Memory vs System Memory
Cache memory is built on the considerably quicker (and more costly) Static RAM, while system memory is built on the slower DRAM (Dynamic RAM). The fundamental distinction between the two is that the former is constructed using CMOS technology and transistors (six for per block), whilst the latter use capacitors and transistors.
To keep data for extended periods of time, DRAM must be regularly updated (due to leaking charges). As a result, it consumes substantially more power and is slower. SRAM does not need to be updated and is thus significantly more efficient. However, the expensive expense has limited its usage to CPU cache, restricting its popular acceptance.
Coherency of Cache
Because caches maintain local copies of data that have their canonical versions in memory, data incoherence arises. For example, if one core modifies data in the core’s private data cache, other cores may miss the change since they access the same memory address via their own caches. Furthermore, if a device alters data in memory, the update may not be instantly visible to any of the CPU cores, since they may be reading stale data from one of the hierarchy’s caches.
Per Core Instruction Cache
We will begin our investigation of GPU per core caches with the shader instruction cache since it is the simplest.
While the programming approach and hence the hardware implementation of GPUs are quite different from those of CPUs, the essential building blocks and methodologies aren’t that distinct. For example, many GPU designs include SIMD processing units, but instead of enabling vector instructions from individual threads, they employ a SIMT (Single Instruction Multiple Thread) execution paradigm in which each lane is effectively executing a different thread (shader invocation).
GPU Cache
This is an excellent approach for GPUs since they often execute very basic programmes over a wide collection of work items, so they benefit from being able to schedule instructions for a group of threads collectively rather than separately. A wave is formed by a number of threads that are co-executed in this SIMT way (typically in lock-step) (wavefront in AMD terminology, warp in NVIDIA terminology, subgroup in Vulkan terminology).
Per Core Data Cache
Typically, each GPU core has one or more specialised shader data caches. Shaders could only read data from memory but not write to it in early GPU architectures since shaders often only required to access texture and buffer inputs to fulfil their functions, hence per core data caches were historically read-only. However, newer systems designed for flexible GPGPU compute processes required distributed writes to memory, necessitating the use of a read/write data cache. These caches often use a write-through policy, which means that writes are promptly propagated to the next cache level.
Cache for the Whole Device
With a few noteworthy exceptions, the per core and various lower level caches of GPUs are supplied by a device wide cache that offers coherent stored data across the cores, and it’s often the final cache level in the hierarchy. This is a read/write cache that often supports atomic operations as well. In fact, all atomic actions on memory-backed data done by shaders/kernels are handled here, bypassing lower level caches in the process. Since a consequence, atomic operations are, by definition, consistent across threads executing on different GPU cores; however, this does not ensure that ordinary readers will receive the results of atomics, as they will continue to be provided by default from per-core data caches.
Caches in other locations
We’ve only discussed caches used to speed memory accesses done by shader cores so far, but GPUs often utilise additional caches that service the many fixed function hardware components that execute certain stages or functions of the graphics pipeline. The vertex attribute cache, which was previously used to store vertex attribute values before dispatching the appropriate vertex shader invocations, is an example of such a cache. Nowadays, due to the increased programmability of the geometry processing stages, there is usually no need for such a dedicated cache because attribute data reads frequently occur from within the shaders, either explicitly or implicitly, especially in modern geometry pipelines using tessellation, geometry shaders, or mesh shaders.
Memory Sharing
On current GPUs, each core has its own shared memory, which is mainly used to communicate data across compute threads (or compute shader invocations) in the same workgroup and even allows atomic operations. Shared memory is not a cache, but rather a form of scratchpad memory, but we include it here because it serves a similar function to caches in that it allows you to retain frequently used data near to the processing units.
