diff --git a/Thoughts-On-Fast-Bitmap-Image-Rendering.md b/Thoughts-On-Fast-Bitmap-Image-Rendering.md deleted file mode 100644 index faba722..0000000 --- a/Thoughts-On-Fast-Bitmap-Image-Rendering.md +++ /dev/null @@ -1,147 +0,0 @@ -### Introduction - -Rendering bitmap images in WebGL is accomplished by issuing -surface geometry that is then textured within the _Fragment_ -_Shader_. - -The WebGL API requires texture memory resources _(Texture_ -_Objects)_ to be assigned to _Texturing_ _Units_ as part of -the rendering state. This limitation is often (unless for -very recent desktop graphic cards) hardware-driven. - -The number of WebGL Texture Objects that can be used within -one _Batch_ is limited. Rendering many different images, -such as icons, each presented to WebGL as a single Texture -Object therefore complicates the rendering process and -requires costly CPU intervention. - -### Tiled Texture - -This problem is typically solved by using a single Texture -Object that represents a tiled image consisting of the -individual images to be rendered and appropriate texture -coordinates to select those sub-images. - -### Caching - -Uploading data to the GPU from the host system is costly (and -intentionally bandwidth-limited for consumer hardware within -PC systems other than for high-end workstation cards without -this limitation). Therefore we want to keep as much data as -possible on the GPU between frames in order to keep up -interactive and very competitive frame rates. - -In order to accomplish this we are building a _Cache_: -Whenever an image is needed, we check whether it has been -uploaded already. If so, we can use it. Otherwise when there -are free tiles we can use one of those tiles to upload it. -If there are no free tiles we have to evict an image that is -not required to render the current batch or frame. Ideally we want -to pick an image that is unlikely to be needed in the next -couple of batches and frames (as could be the case just using -_RNU_). _LRU_ and _LFU_ strategies are suitable for more or less -repetitively used images, respectively. - -Whenever images are evicted from the cache, rendering data -referencing it becomes invalid. - -We assume the cache uses one tile size for all images (that -may or may not use the entire space provided by the tile). - -### Differently Sized Tiles - -Using just a single tile size may waste too much GPU memory -when image sizes differ too much. In the following, we briefly -discuss four alternatives that could be used to approach this -problem. Those are (ordered by sophistication and implementation -complexity): - -1. Instantiation of multiple image caches (each for a predefined - image size) using different Texture Objects, -2. Instantiation of multiple image caches (each for a predefined - image size) using different areas of the same Texture Object, -3. Using layered image caches, where the area used by one is - allocated from a layer for larger-sized tiles (again using - predefined image sizes, but dynamically adjusting to the - needs) -4. Implementing one image cache that allows allocation of - relatively small-sized, consecutive tiles that performs - defragmentation so larger blocks are kept available. - -The first two approaches are trivial to implement. Assigning -Texture regions to caches manually can be impractical on the -maintenance side, however, allowing it to be possible can be -useful in order to evolve towards alternative three. - -The third approach can -to some extent- dynamically adjust to -the image sizes needed by the application. When images come in -widely varying sizes at unforeseeable number, it is clearly -much better than the previously discussed alternatives. Care must -be taken to avoid that underpopulated sub-caches occupy too -many large tiles (the kind of fragmentation encountered with -this allocation scheme) e.g. by forcibly removing underpopulated -sub-caches when there is enough free space for tiles of their -size elsewhere. - -The fourth approach most effectively manages memory but carries -a significant overhead for defragmentation (while moving the data -can be done on the GPU, the CPU has to determine the permutation, -reordering tiles in Z-curve arrangement according to the eviction -criterion). Also the texture coordinates used for rendering have -to be updated, requiring the coordinates to be uploaded again. - -### Notes on Broader Applicability - -Not every raster image must necessarily originate from an URL: -Labels or individual Unicode characters may be rendered on the -client side. The API of the image cache should make no assumptions -about how the image is generated or uploaded (we're also able to -use the texture as a render target, then). -Furthermore images may be encoded as scalable, monochrome images -(very attractive for text, maybe for "instancing vector graphics" -as well, see -[this paper](http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf)) -that occupy just one or two 8-bit color channels of the texture - -we might want to allow sharing tiles in this case in the future. - -### Glossary - -***GPU*** - **G**raphics **P**rocessing **U**nit - -***CPU*** - **C**entral **P**rocessing **U**nit; processor -steering a computer - -***Shader*** - a routine that runs at a specific stage of a -programmable rendering pipeline of a _GPU_ - -***Fragment Shader*** - the routine within the currently active -graphics pipeline routine that is executed on a per-pixel basis - -***Texturing Unit*** - custom hardware within the _GPU_ for -addressing, sampling and converting input data from a -certain region of memory represented by a _Texture_ _Object_. - -***Texture Object*** - a certain, uniquely identifiable region of -_Texture_ _Memory_ associated with a 1-3 dimensional image of -data tuples - -***Texture Memory*** - a region of a GPU's memory that is optimized -for 2D spatial access patterns - -***Batch (OpenGL Rendering)*** - bulk-processed data to be rendered -under a shared rendering state - -***Cache*** - quickly accessible storage for data that has been -acquired in the past (typically from slower memory) and is now -kept ready for future access - -***Eviction Strategy (Caching)*** - determination of data to be removed -from a full _Cache_ when space for new data is needed - -***RNU (Eviction Strategy for Caching)*** - **R**ecently **N**ot **U**sed - -***LRU (Eviction Strategy for Caching)*** - **L**east **R**ecently **U**sed - -***LFU (Eviction Strategy for Caching)*** - **L**east **F**requently **U**sed - -