Masking in GLSL

WebGL is complicated. Libraries like Three.js have made it more approachable but to do advanced effects you still need to understand the inner workings of WebGL, this includes GLSL shaders. Shaders are small programs that run on the GPU and they consist of two types, vertex shaders which draw the geometry and fragment shaders which draw the colors to present the final output.

To mask content or manipulate the transparency of items in WebGL we are going to leverage the fragment shader. Because WebGL and Three.js are big topics I will cover how to use custom fragment shaders another time. For managing scope we will assume you already know how to do this and focus on the the shaders.

To familiarize yourself with shaders I recommend checking out or shadertoy which are web applications that do live GLSL rendering. We won’t be using them here because they don’t support alpha. This is because everything is done in one shader, so it would influence the transparency of everything. But they are a good way to play around with GLSL and see what others can achieve.

GLSL is similar to C/C++ with strict types and operators. The fragment shader which influences our color channels consists of a main method which sets a variable named gl_FragColor of type vec4 (a vector containing 4 floats), representing the red, green, blue, and alpha channels being drawn on the vertices. The following example returns the color red:

void main() {
     gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);

To manipulate the alpha channel all we have to do is change that last value of the vec4. So if we set it to 0.5 the returned color will be 50% transparent.

To use images as masks we need to pass the texture along to our shaders.

Vertex Shader

varying vec2 vUv;

void main() {
  vUv = uv;
  gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);

Fragment Shader

varying vec2 vUv;
uniform sampler2D texture1;

void main() {
  vec4 texture = texture2D(texture1, vUv);
  gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0-texture.r);

This requires that we pass some data from our vertex shader into our fragment shader. We do this by creating a variable to store the UV’s. Then we access those UV’s in our fragment shader and pass in our image as a Sampler2D. Then we call the method texture2D() which takes the Sampler2D and the UV coordinates and returns a vec4. We can then use a color channel of that texture and apply it to the alpha channel, in this case the red channel.

You can see more details and examples of masking in GLSL in the video above. Below is one of the examples hosted on codepen.

See the Pen Masking #4: Text by CJ Gammon (@cjgammon) on CodePen.

Click here to download the demo.