Created Thoughts On Fast Bitmap Image Rendering (markdown)

2013-07-20 16:39:40 -07:00
parent ba2a1b3f38
commit eb080834b4
1 changed files with 147 additions and 0 deletions
@@ -0,0 +1,147 @@
 ### Introduction
 Rendering bitmap images in WebGL is accomplished by issuing
 surface geometry that is then textured within the _Fragment_
 _Shader_. 
 The WebGL API requires texture memory resources _(Texture_ 
 _Objects)_ to be assigned to _Texturing_ _Units_ as part of 
 the rendering state. This limitation is often (unless for 
 very recent desktop graphic cards) hardware-driven.
 The number of WebGL Texture Objects that can be used within 
 one  _Batch_ is limited. Rendering many different images,
 such as icons, each presented to WebGL as a single Texture 
 Object therefore complicates the rendering process and 
 requires costly CPU intervention.
 ### Tiled Texture
 This problem is typically solved by using a single Texture
 Object that represents a tiled image consisting of the
 individual images to be rendered and appropriate texture
 coordinates to select those sub-images.
 ### Caching
 Uploading data to the GPU from the host system is costly (and
 intentionally bandwidth-limited for consumer hardware within
 PC systems other than for high-end workstation cards without 
 this limitation). Therefore we want to keep as much data as
 possible on the GPU between frames in order to keep up 
 interactive and very competitive frame rates.
 In order to accomplish this we are building a _Cache_: 
 Whenever an image is needed, we check whether it has been
 uploaded already. If so, we can use it. Otherwise when there
 are free tiles we can use one of those tiles to upload it.
 If there are no free tiles we have to evict an image that is 
 not required to render the current batch or frame. Ideally we want
 to pick an image that is unlikely to be needed in the next
 couple of batches and frames (as could be the case just using 
 _RNU_). _LRU_ and _LFU_ strategies are suitable for more or less
 repetitively used images, respectively.
 Whenever images are evicted from the cache, rendering data
 referencing it becomes invalid. 
 We assume the cache uses one tile size for all images (that 
 may or may not use the entire space provided by the tile).
 ### Differently Sized Tiles
 Using just a single tile size may waste too much GPU memory
 when image sizes differ too much. In the following, we briefly
 discuss four alternatives that could be used to approach this 
 problem. Those are (ordered by sophistication and implementation 
 complexity):
 1. Instantiation of multiple image caches (each for a predefined
   image size) using different Texture Objects,
 2. Instantiation of multiple image caches (each for a predefined
   image size) using different areas of the same Texture Object,
 3. Using layered image caches, where the area used by one is
   allocated from a layer for larger-sized tiles (again using
   predefined image sizes, but dynamically adjusting to the
   needs)
 4. Implementing one image cache that allows allocation of
   relatively small-sized, consecutive tiles that performs
   defragmentation so larger blocks are kept available.
 The first two approaches are trivial to implement. Assigning
 Texture regions to caches manually can be impractical on the
 maintenance side, however, allowing it to be possible can be 
 useful in order to evolve towards alternative three.
 The third approach can -to some extent- dynamically adjust to
 the image sizes needed by the application. When images come in
 widely varying sizes at unforeseeable number, it is clearly
 much better than the previously discussed alternatives. Care must
 be taken to avoid that underpopulated sub-caches occupy too
 many large tiles (the kind of fragmentation encountered with
 this allocation scheme) e.g. by forcibly removing underpopulated
 sub-caches when there is enough free space for tiles of their
 size elsewhere.
 The fourth approach most effectively manages memory but carries
 a significant overhead for defragmentation  (while moving the data
 can be done on the GPU, the CPU has to  determine the permutation,
 reordering tiles in Z-curve arrangement according to the eviction 
 criterion). Also the texture coordinates used for rendering have
 to be updated, requiring the coordinates to be uploaded again.
 ### Notes on Broader Applicability
 Not every raster image must necessarily originate from an URL:
 Labels or individual Unicode characters may be rendered on the
 client side. The API of the image cache should make no assumptions
 about how the image is generated or uploaded (we're also able to
 use the texture as a render target, then).
 Furthermore images may be encoded as scalable, monochrome images 
 (very attractive for text, maybe for "instancing vector graphics"
 as well, see
 [this paper](http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf))
 that occupy just one or two 8-bit color channels of the texture -
 we might want to allow sharing tiles in this case in the future.
 ### Glossary
 ***GPU*** - **G**raphics **P**rocessing **U**nit
 ***CPU*** - **C**entral **P**rocessing **U**nit; processor 
 steering a computer
 ***Shader*** - a routine that runs at a specific stage of a
 programmable rendering pipeline of a _GPU_ 
 ***Fragment Shader*** - the routine within the currently active 
 graphics pipeline routine that is executed on a per-pixel basis
 ***Texturing Unit*** - custom hardware within the _GPU_ for 
 addressing, sampling and converting input data from a 
 certain region of memory represented by a _Texture_ _Object_.
 ***Texture Object*** - a certain, uniquely identifiable region of 
 _Texture_ _Memory_  associated with a 1-3 dimensional image of
 data tuples 
 ***Texture Memory*** - a region of a GPU's memory that is optimized
 for 2D spatial access patterns
 ***Batch (OpenGL Rendering)*** - bulk-processed data to be rendered
 under a shared rendering state
 ***Cache*** - quickly accessible storage for data that has been 
 acquired in the past (typically from slower memory) and is now
 kept ready for future access
 ***Eviction Strategy (Caching)*** - determination of data to be removed
 from a full _Cache_ when space for new data is needed
 ***RNU (Eviction Strategy for Caching)*** - **R**ecently **N**ot **U**sed 
 ***LRU (Eviction Strategy for Caching)*** - **L**east **R**ecently **U**sed
 ***LFU (Eviction Strategy for Caching)*** - **L**east **F**requently **U**sed