Fractal fun with WebGL
Exploring fractals with WebGL and React
Here I'll demonstrate some of my experiments with WebGL and drawing fractals. If you'd like to see what I've built, there is in interactive fractal explorer at the bottom of this page. If you'd like to learn more about fractals and the math involved, I'd recommend starting by reviewing imaginary numbers and researching the Mandelbrot set.
For those who are unfamiliar, WebGL gives us a way to run code on the GPU in the web browser environment. I highly recommend reading this page to learn more.
Running code on the GPU requires a canvas element, data, and a shader program. JavaScript's job is to initialize the graphics library, pass data to the shader program, and connect the shader program to a canvas element. The shader program consists of the vertex shader, which positions the vertices in some geometry and passes the result to the fragment shader, which determines the color of a given pixel.
So how do we draw fractals with a shader program? The first thing to do is create some vertex coordinates. I'm just using a 2d plane for this. Since I'm using a TRIANGLE_STRIP, I just pass the coordinates of a square composed of two triangles ([-1, -1, -1, 1, 1, -1, 1, 1]) to the vertex shader which does not modify these values. This array represents 4 (x, y) points.
#version 300 es
in vec2 a_position;
void main() {
gl_Position = vec4(a_position, 0, 1);
}The fragment shader will just implement an algorithm for determining whether a pixel at some coordinates is within the Mandelbrot set. Values managed by UI controls on the page are passed to the fragment shader with uniforms. I'm using a pretty simple coloring algorithm here and changing the result based on the current time.
#version 300 es
precision highp float;
uniform int u_max_i;
uniform float u_time;
uniform float u_zoom;
uniform float u_x_pos;
uniform float u_y_pos;
uniform vec2 u_resolution;
const int COLOR_BUCKETS = 128;
out vec4 outColor;
void main() {
int i = 0;
// subtracting the resolution/2.0 causes the zoom focal point to be the center of the canvas.
// subtracting the position divided by resolution is what moves the fractal around when panning.
float x0 = (gl_FragCoord.x - u_resolution.x/2.0)/u_resolution.x/u_zoom - u_x_pos/u_resolution.x;
float y0 = (gl_FragCoord.y - u_resolution.y/2.0)/u_resolution.y/u_zoom - u_y_pos/u_resolution.y;
float x = 0.0;
float y = 0.0;
float xtemp = 0.0;
float x_square = 0.0;
float y_square = 0.0;
while (x_square + y_square <= 4.0 && i < u_max_i) {
xtemp = x_square - y_square + x0;
y = 2.0 * x * y + y0;
x = xtemp;
x_square = x * x;
y_square = y * y;
i++;
}
// bucket the escape velocity and normalize value to 1.0
float magnitude = float(i % COLOR_BUCKETS) / float(COLOR_BUCKETS);
if (i < 1 || i == u_max_i) {
outColor = vec4(vec3(0.0), 1.0);
} else {
outColor = vec4(
0.0,
abs(sin(magnitude + u_time / 2100.0)),
abs(cos(magnitude + u_time / 1900.0)) * 0.7,
1.0
);
}
}Of course I built the "Fractal Explorer" (and this entire website) in React, because React is awesome. As far as WebGL is concerned here, I've simply made a component that renders out a canvas and takes props including vertex shader and fragment shader code. The shader program is compiled and rendered when this component mounts.
import React, { FC, useRef, useEffect, useCallback, useState } from "react";
import cn from "classnames";
import {
getGl,
createProgram,
render,
createBindVao,
enableVertexAttribute
} from "graphics/gl";
interface ShaderProgramRendererProps {
className?: string;
animate?: boolean;
onMouseDown?: (e: React.MouseEvent) => void;
vertexSource: string;
fragmentSource: string;
vertexData: Array<VertexAttributeData>;
setUniforms?: (gl: WebGL2RenderingContext, program: WebGLProgram) => void;
}
export const ShaderProgramRenderer: FC<ShaderProgramRendererProps> = ({
className = "",
animate,
onMouseDown = () => {},
vertexSource,
fragmentSource,
vertexData,
setUniforms = () => {}
}) => {
const canvas = useRef<HTMLCanvasElement>(null);
const gl = useRef<WebGL2RenderingContext | null>(null);
const program = useRef<WebGLProgram | null>(null);
const vaoRef = useRef<WebGLVertexArrayObject | null>(null);
const vertexCountRef = useRef<number>(0);
const rafFrame = useRef(0);
const [hasError, setHasError] = useState(false);
const renderCallback = useCallback(
(time: number) => {
if (canvas.current && gl.current && program.current) {
render({
gl: gl.current,
program: program.current,
time,
setUniforms,
vao: vaoRef.current,
vertexCount: vertexCountRef.current
});
if (animate) {
rafFrame.current = requestAnimationFrame(renderCallback);
}
}
},
[animate, setUniforms]
);
// init shader program and required refs
useEffect(() => {
if (canvas.current) {
try {
gl.current = getGl(canvas.current);
program.current = createProgram(
gl.current,
vertexSource,
fragmentSource
);
const vao = createBindVao(gl.current);
vaoRef.current = vao;
vertexData.forEach(data =>
enableVertexAttribute(
gl.current as WebGL2RenderingContext,
program.current as WebGLProgram,
data
)
);
vertexCountRef.current =
vertexData.length > 0 ? vertexData[0].count : 0;
} catch (e) {
setHasError(true);
}
}
}, [vertexSource, fragmentSource, setHasError, vertexData]);
// initial render effect
useEffect(() => {
rafFrame.current = requestAnimationFrame(renderCallback);
return () => cancelAnimationFrame(rafFrame.current);
// a new shader program should be rendered if either shader source updates
}, [renderCallback, vertexSource, fragmentSource]);
return (
<div
className={cn(className, "shader-program-renderer", {
"has-error": hasError
})}
onMouseDown={onMouseDown}
>
{!hasError && <canvas ref={canvas} />}
{hasError && <strong>Unable to initialize WebGL on this device.</strong>}
</div>
);
};
export default ShaderProgramRenderer;This works quite well for creating high performance graphics at low levels of zoom. However, pixellation will occur before you can zoom in very far. This occurs because of a lack of precision in the floating point variables used to do the calculations.
If you want to really push the limits of what you can do though, it is possible to quadruple the available precision by representing values as 4 dimensional vectors of floats. Thanks to Henry Thasler's article about this technique, I was able to implement this approach in the high-res version of the mandelbrot explorer below.
The actual precision available is hardware dependent, so some pixellation may still occur even with this technique. Similar to the pixellation issue, it may become quite difficult to pan at high levels of zoom. This is because there is a loss of precision when JavaScript passes the pan coordinates to WebGL. This could be improved by representing the JavaScript pan coordinates as 4 dimensional vectors of values before passing them to WebGL.