Destroyed Thoughts On Fast Bitmap Image Rendering (markdown)
Tim Schaub
@@ -1,147 +0,0 @@
|
||||
### Introduction
|
||||
|
||||
Rendering bitmap images in WebGL is accomplished by issuing
|
||||
surface geometry that is then textured within the _Fragment_
|
||||
_Shader_.
|
||||
|
||||
The WebGL API requires texture memory resources _(Texture_
|
||||
_Objects)_ to be assigned to _Texturing_ _Units_ as part of
|
||||
the rendering state. This limitation is often (unless for
|
||||
very recent desktop graphic cards) hardware-driven.
|
||||
|
||||
The number of WebGL Texture Objects that can be used within
|
||||
one _Batch_ is limited. Rendering many different images,
|
||||
such as icons, each presented to WebGL as a single Texture
|
||||
Object therefore complicates the rendering process and
|
||||
requires costly CPU intervention.
|
||||
|
||||
### Tiled Texture
|
||||
|
||||
This problem is typically solved by using a single Texture
|
||||
Object that represents a tiled image consisting of the
|
||||
individual images to be rendered and appropriate texture
|
||||
coordinates to select those sub-images.
|
||||
|
||||
### Caching
|
||||
|
||||
Uploading data to the GPU from the host system is costly (and
|
||||
intentionally bandwidth-limited for consumer hardware within
|
||||
PC systems other than for high-end workstation cards without
|
||||
this limitation). Therefore we want to keep as much data as
|
||||
possible on the GPU between frames in order to keep up
|
||||
interactive and very competitive frame rates.
|
||||
|
||||
In order to accomplish this we are building a _Cache_:
|
||||
Whenever an image is needed, we check whether it has been
|
||||
uploaded already. If so, we can use it. Otherwise when there
|
||||
are free tiles we can use one of those tiles to upload it.
|
||||
If there are no free tiles we have to evict an image that is
|
||||
not required to render the current batch or frame. Ideally we want
|
||||
to pick an image that is unlikely to be needed in the next
|
||||
couple of batches and frames (as could be the case just using
|
||||
_RNU_). _LRU_ and _LFU_ strategies are suitable for more or less
|
||||
repetitively used images, respectively.
|
||||
|
||||
Whenever images are evicted from the cache, rendering data
|
||||
referencing it becomes invalid.
|
||||
|
||||
We assume the cache uses one tile size for all images (that
|
||||
may or may not use the entire space provided by the tile).
|
||||
|
||||
### Differently Sized Tiles
|
||||
|
||||
Using just a single tile size may waste too much GPU memory
|
||||
when image sizes differ too much. In the following, we briefly
|
||||
discuss four alternatives that could be used to approach this
|
||||
problem. Those are (ordered by sophistication and implementation
|
||||
complexity):
|
||||
|
||||
1. Instantiation of multiple image caches (each for a predefined
|
||||
image size) using different Texture Objects,
|
||||
2. Instantiation of multiple image caches (each for a predefined
|
||||
image size) using different areas of the same Texture Object,
|
||||
3. Using layered image caches, where the area used by one is
|
||||
allocated from a layer for larger-sized tiles (again using
|
||||
predefined image sizes, but dynamically adjusting to the
|
||||
needs)
|
||||
4. Implementing one image cache that allows allocation of
|
||||
relatively small-sized, consecutive tiles that performs
|
||||
defragmentation so larger blocks are kept available.
|
||||
|
||||
The first two approaches are trivial to implement. Assigning
|
||||
Texture regions to caches manually can be impractical on the
|
||||
maintenance side, however, allowing it to be possible can be
|
||||
useful in order to evolve towards alternative three.
|
||||
|
||||
The third approach can -to some extent- dynamically adjust to
|
||||
the image sizes needed by the application. When images come in
|
||||
widely varying sizes at unforeseeable number, it is clearly
|
||||
much better than the previously discussed alternatives. Care must
|
||||
be taken to avoid that underpopulated sub-caches occupy too
|
||||
many large tiles (the kind of fragmentation encountered with
|
||||
this allocation scheme) e.g. by forcibly removing underpopulated
|
||||
sub-caches when there is enough free space for tiles of their
|
||||
size elsewhere.
|
||||
|
||||
The fourth approach most effectively manages memory but carries
|
||||
a significant overhead for defragmentation (while moving the data
|
||||
can be done on the GPU, the CPU has to determine the permutation,
|
||||
reordering tiles in Z-curve arrangement according to the eviction
|
||||
criterion). Also the texture coordinates used for rendering have
|
||||
to be updated, requiring the coordinates to be uploaded again.
|
||||
|
||||
### Notes on Broader Applicability
|
||||
|
||||
Not every raster image must necessarily originate from an URL:
|
||||
Labels or individual Unicode characters may be rendered on the
|
||||
client side. The API of the image cache should make no assumptions
|
||||
about how the image is generated or uploaded (we're also able to
|
||||
use the texture as a render target, then).
|
||||
Furthermore images may be encoded as scalable, monochrome images
|
||||
(very attractive for text, maybe for "instancing vector graphics"
|
||||
as well, see
|
||||
[this paper](http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf))
|
||||
that occupy just one or two 8-bit color channels of the texture -
|
||||
we might want to allow sharing tiles in this case in the future.
|
||||
|
||||
### Glossary
|
||||
|
||||
***GPU*** - **G**raphics **P**rocessing **U**nit
|
||||
|
||||
***CPU*** - **C**entral **P**rocessing **U**nit; processor
|
||||
steering a computer
|
||||
|
||||
***Shader*** - a routine that runs at a specific stage of a
|
||||
programmable rendering pipeline of a _GPU_
|
||||
|
||||
***Fragment Shader*** - the routine within the currently active
|
||||
graphics pipeline routine that is executed on a per-pixel basis
|
||||
|
||||
***Texturing Unit*** - custom hardware within the _GPU_ for
|
||||
addressing, sampling and converting input data from a
|
||||
certain region of memory represented by a _Texture_ _Object_.
|
||||
|
||||
***Texture Object*** - a certain, uniquely identifiable region of
|
||||
_Texture_ _Memory_ associated with a 1-3 dimensional image of
|
||||
data tuples
|
||||
|
||||
***Texture Memory*** - a region of a GPU's memory that is optimized
|
||||
for 2D spatial access patterns
|
||||
|
||||
***Batch (OpenGL Rendering)*** - bulk-processed data to be rendered
|
||||
under a shared rendering state
|
||||
|
||||
***Cache*** - quickly accessible storage for data that has been
|
||||
acquired in the past (typically from slower memory) and is now
|
||||
kept ready for future access
|
||||
|
||||
***Eviction Strategy (Caching)*** - determination of data to be removed
|
||||
from a full _Cache_ when space for new data is needed
|
||||
|
||||
***RNU (Eviction Strategy for Caching)*** - **R**ecently **N**ot **U**sed
|
||||
|
||||
***LRU (Eviction Strategy for Caching)*** - **L**east **R**ecently **U**sed
|
||||
|
||||
***LFU (Eviction Strategy for Caching)*** - **L**east **F**requently **U**sed
|
||||
|
||||
|
||||
Reference in New Issue
Block a user