Defining custom pipelines

GStreamer is based around the idea of pipeline: a sequence of interconnected elements that handle the video data from it source (a file or a capture device) to the final output (screen, disk). A GStreamer pipeline can be built from the command line with the gst-launch tool:

> gst-launch videotestsrc pattern=0 ! ffmpegcolorspace ! 
             video/x-raw-rgb, width=320, height=240, bpp=32, depth=24 ! xvimagesink

GSVideo has a class that allows to build custom pipelines like this, which is called GSCustom:

import codeanticode.gsvideo.*;

GSCustom pipe;

void setup() {
  size(640, 480);
  
  // VideoTestSrc pipeline  
  pipe = new GSCustom(this, "videotestsrc pattern=0 ! ffmpegcolorspace !" + 
                            "video/x-raw-rgb, width=320, height=240, bpp=32, depth=24");
}

void draw() {
  if (pipe.available() == true) {
    pipe.read();
    image(pipe, 160, 100);
  }
}

In fact, all that the other GSVideo objects do is to create a specific pipeline (for movie playback, video capture, etc).

GStreamer works better in combination with Processing than Quicktime, and this makes possible to play high resolution video files through the GSMovie object. However, for this to be possible with HD movies, the rendering of the video frames has to be accelerated through OpenGL. The GLGraphics library makes easier the handling of OpenGL textures inside Processing. The method consists in reading the frames with the GSMovie object, and then copying the movie pixels into a GLTexture object which is then rendered to the screen:

import processing.opengl.*;

import codeanticode.gsvideo.*;
import codeanticode.glgraphics.*;

GSMovie movie;
GLTexture tex;

void setup() {
  size(640, 480, GLConstants.GLGRAPHICS);
  
  movie = new GSMovie(this, "hdclip.mov");
  movie.loop();
   
  tex = new GLTexture(this);
}

void movieEvent(GSMovie movie) {
  movie.read();
}

void draw() {
  background(0);

  if ((1 < movie.width) && (1 < movie.height)) {

    // Copies the latest movie frame into the texture tex.
    tex.putPixelsIntoTexture(movie);

    image(tex, 0, 0, width, height);
  }
}