Introduction

Examples

Advanced use

GPGPU

Documentation

Installation

Downloads

Browse source

Forums

Tracker

Wiki

Blog
Using the GLTexture class

GLTexture adds support for OpenGL textures to the built-in PImage class. The texture can be thought in a similar manner as the pixels property of PImage. An image is copied from the canvas of the PImage to the pixels array by calling the loadPixels() function. Similarly, the pixel data can be transferred to the texture by calling loadTexture(). In a reverse order, the updateTexture() copies the texture data to the pixels property, while updatePixels() puts the pixels into the drawing canvas.

So for instance, the following code is a valid example of drawing operation (in this case, clearing the texture) whose result is taken down to the PImage canvas by succesively calling updateTexture() and then updatePixels(): 
import processing.opengl.*;
import codeanticode.glgraphics.*;

GLTexture tex;

void setup()
{
    size(640, 480, OPENGL);
    
    // Creates a texture object of 200x200 pixels.
    tex = new GLTexture(this, 200, 200);
}

void draw()
{
    background(0);
  
    // Draws a random color to the texture.
    tex.clear(random(0, 255), random(0, 255), random(0, 255), 255);
    
    tex.updateTexture();
    tex.updatePixels();
    
    image(tex, mouseX, mouseY);
}

However, a problem of this approach is that transfering texture data from the GPU memory to the pixels array on CPU memory could slow down the program, specially for large textures, and it should be minimized. In order to solve this problem, the library includes a new renderer that can draw textures directly. This renderer is initialized by calling:

import processing.opengl.*;
import codeanticode.glgraphics.*;

GLTexture tex;

void setup()
{
    size(640, 480, GLConstants.GLGRAPHICS);
    ...
The GLGraphics renderer understands that the GLTexture object contains an OpenGL texture and renders it directly through the GPU, which is much faster than using the regular PImage canvas. In this case, there is no need to call neither updateTexture() nor updatePixels(). In fact, any drawing operation involving a GLTexture object will be accelerated through the video card when using the GLGraphics renderer.

A GLTexture can be initialized in different ways: 
// Loading an image directly upon creation:
tex = new GLTexture(this, "image.jpg");

// Creating a texture of a given size:
tex = new GLTexture(this, 200, 200);

// It can be created empty and initialized later:
tex = new GLTexture(this);
...
tex.init(200, 200);
...
tex.loadTexture("image.jpg");

// Created with a given size, and then using the 
// pixels array to put data in it:
tex = new GLTexture(this, 100, 100);
tex.loadPixels();
for (int i = 0; i < 200; i++) tex.pixels[i] = 0xff000000;
tex.loadTexture();
The GLTexture and PImage classes can interact with each other to transfer image data back and forth:
PImage img;
GLTexture tex;
...

// Loading a PImage and then copying into a GLTexture:
img = loadImage("image.jpg");
tex.putImage(img);

// Copying the texture into a PImage.
tex.getImage(img);
image(img, 0, 0)

Texture filters

A texture filter, encapsulated in the GLTextureFilter class, is basically a GPU shader that operates on an input texture or group of textures and writes the output of the render to another texture(s). The shader can be though as  a computational kernel that is executed on the GPU, such as a gaussian blur or emboss effect. On advantage of this approach is that the calculation is actually off-loaded to the GPU, freeing CPU resources for other operations, such as handling user interaction, general flow of the program, etc.

The configuration of a filter is stored in a xml file, where the names of the shaders that define the filter are stored. An entire shader program can consist in a vertex, geometry and fragment shaders, corresponding to each one of the programmable stages of modern GPUs. At this point, the shaders have to written in the OpenGL Shading Language (GLSL or GLslang). Future releases of the library will include support for the Cg shading language from Nvidia.

The way a texture filter works follows the Processing API for the built-in image filters: 
import processing.opengl.*;
import codeanticode.glgraphics.*;

size(200, 200, GLConstants.GLGRAPHICS);
   
// Initializing source and destination textures.
GLTexture texSrc = new GLTexture(this, "image.jpg");
GLTexture texDest = new GLTexture(this, texSrc.width, texSrc.height);

// Initializing filter.
GLTextureFilter blur = new GLTextureFilter(this, "blur.xml");

// Applying the filter on texture texSrc
// and writing the result to texDest.
texSrc.filter(blur, texDest);

// Drawing the resulting image.
image(texDest, 0, 0, width, height);
The xml file just lists the filename of the fragment shader that contains the gaussian kernel, same basic description strings and the number of input and output textures supported by the filter:
<filter name="gaussian blur">
    <description>3x3 Gaussian blur convolution kernel</description>
    <fragment>blur.glsl</fragment>
    <textures input="1" output="1"></textures>
</filter>
The shader blur.glsl contains the code that is executed by the GPU on each incoming fragment that will be rendered on the output texture:
uniform sampler2D src_tex_unit0;
uniform vec2 src_tex_offset0;

void main(void)
{
	float dx = src_tex_offset0.s;
	float dy = src_tex_offset0.t;
	vec2 st = gl_TexCoord[0].st;

	// Getting colors of the center and surrounding texels.
	vec4 color	 = 4.0 * texture2D(src_tex_unit0, st);
	color		+= 2.0 * texture2D(src_tex_unit0, st + vec2(+dx, 0.0));
	color		+= 2.0 * texture2D(src_tex_unit0, st + vec2(-dx, 0.0));
	color		+= 2.0 * texture2D(src_tex_unit0, st + vec2(0.0, +dy));
	color		+= 2.0 * texture2D(src_tex_unit0, st + vec2(0.0, -dy));
	color		+= texture2D(src_tex_unit0, st + vec2(+dx, +dy));
	color		+= texture2D(src_tex_unit0, st + vec2(-dx, +dy));
	color		+= texture2D(src_tex_unit0, st + vec2(-dx, -dy));
	color		+= texture2D(src_tex_unit0, st + vec2(+dx, -dy));
	
        // Output color is the average.
	gl_FragColor = color / 16.0;
}
The uniform variables src_tex_unit0 and src_tex_offset0 represent the first texture unit and the offset of that texture. These names are a convention that has to be followed by the glsl shaders used in the filters, otherwise the filter cannot send the texture data to the shader (i.e.: a second input texture has to be called src_tex_unit1, and so on). Shader programming is a big topic in itself and it won't be covered in these pages. There is plenty of material online (tutorials, guides, programming resources, etc.) and here there are some useful links:
  GLGraphics library for Processing :: Andrés Colubri 2008