Canvas and WebGL rendering

Overview

Helios supports two rendering paths:

DOM path - Screenshots HTML/CSS/SVG via Playwright
Canvas path - Direct frame encoding via WebCodecs (faster)

For canvas-heavy compositions (Three.js, Pixi.js, raw Canvas API), the canvas path provides significantly better performance through hardware-accelerated encoding.

Automatic path detection

Helios automatically selects the rendering path:

// DOM path (screenshots)
helios.subscribe((state) => {
  document.querySelector('.box').style.opacity = state.time;
});

// Canvas path (WebCodecs)
helios.subscribe((state) => {
  ctx.clearRect(0, 0, width, height);
  ctx.fillRect(state.time * 100, 100, 50, 50);
});

The renderer inspects the composition and selects the optimal strategy.

Canvas rendering path

The canvas path encodes frames directly from a canvas element using the WebCodecs API.

How it works

Frame capture - Renderer calls your draw function
Canvas export - Pixels extracted via canvas.toBlob() or ImageBitmap
Hardware encoding - VideoEncoder compresses frame (GPU-accelerated)
FFmpeg muxing - Frames piped to FFmpeg for MP4 container

Basic setup

import { Helios } from '@helios-project/core';

const canvas = document.getElementById('canvas') as HTMLCanvasElement;
const ctx = canvas.getContext('2d')!;

const helios = new Helios({ duration: 5, fps: 30 });

helios.subscribe((state) => {
  const { currentFrame, fps, duration } = state;
  const progress = currentFrame / (duration * fps);
  
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  
  // Draw based on progress
  ctx.fillStyle = `hsl(${progress * 360}, 100%, 50%)`;
  ctx.fillRect(100, 100, 200, 200);
});

Three.js optimization

Basic Three.js composition

import * as THREE from 'three';
import { Helios } from '@helios-project/core';

const canvas = document.getElementById('canvas') as HTMLCanvasElement;
const renderer = new THREE.WebGLRenderer({ 
  canvas, 
  antialias: true,
  alpha: false // Disable if no transparency needed
});

const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, 16/9, 0.1, 1000);
camera.position.z = 5;

// Add objects
const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshStandardMaterial({ color: 0x00ff00 });
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);

// Lighting
const light = new THREE.DirectionalLight(0xffffff, 1);
light.position.set(1, 1, 1);
scene.add(light);

const helios = new Helios({ duration: 10, fps: 30 });

helios.subscribe((state) => {
  const time = state.currentTime;
  
  // Animate based on time (not delta)
  cube.rotation.x = time * 0.5;
  cube.rotation.y = time * 0.8;
  
  renderer.render(scene, camera);
});

Performance tips for Three.js

Use static scene graph

// Good: Modify existing objects
cube.rotation.y = time;

// Bad: Create new objects every frame
scene.add(new THREE.Mesh(geometry, material)); // Memory leak!

Disable auto-clear for layers

renderer.autoClear = false;

// Manual render order
renderer.clear();
renderer.render(backgroundScene, camera);
renderer.clearDepth();
renderer.render(foregroundScene, camera);

Reuse geometries and materials

const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshStandardMaterial({ color: 0xff0000 });

for (let i = 0; i < 100; i++) {
  const mesh = new THREE.Mesh(geometry, material);
  scene.add(mesh);
}

Use instancing for repeated objects

import { InstancedMesh } from 'three';

const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshStandardMaterial();
const count = 1000;

const instancedMesh = new THREE.InstancedMesh(geometry, material, count);

const matrix = new THREE.Matrix4();
for (let i = 0; i < count; i++) {
  matrix.setPosition(Math.random() * 10, Math.random() * 10, Math.random() * 10);
  instancedMesh.setMatrixAt(i, matrix);
}

scene.add(instancedMesh);

Optimize shadow maps

renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap; // Better quality
renderer.shadowMap.autoUpdate = false; // Update manually if static

// Only cast shadows from key lights
directionalLight.castShadow = true;
directionalLight.shadow.mapSize.width = 2048;
directionalLight.shadow.mapSize.height = 2048;

Pixi.js rendering

Pixi.js is a 2D WebGL rendering library optimized for sprites and 2D graphics.

import * as PIXI from 'pixi.js';
import { Helios } from '@helios-project/core';

const app = new PIXI.Application({
  view: document.getElementById('canvas') as HTMLCanvasElement,
  width: 1920,
  height: 1080,
  backgroundColor: 0x1099bb,
});

const sprite = PIXI.Sprite.from('assets/bunny.png');
sprite.anchor.set(0.5);
sprite.x = app.screen.width / 2;
sprite.y = app.screen.height / 2;
app.stage.addChild(sprite);

const helios = new Helios({ duration: 5, fps: 30 });

helios.subscribe((state) => {
  const progress = state.currentFrame / (state.duration * state.fps);
  
  // Animate sprite
  sprite.rotation = progress * Math.PI * 2;
  sprite.scale.set(1 + Math.sin(progress * Math.PI * 2) * 0.2);
  
  // Manually render (disable auto-render)
  app.renderer.render(app.stage);
});

// Disable Pixi's auto-render ticker
app.ticker.stop();

GPU acceleration setup

GPU acceleration is critical for rendering performance.

Browser configuration

When using @helios-project/renderer, GPU acceleration is enabled by default:

import { Renderer } from '@helios-project/renderer';

const renderer = new Renderer({
  mode: 'canvas',
  browserConfig: {
    headless: true,
    args: [
      '--use-gl=egl',              // Force OpenGL backend
      '--ignore-gpu-blocklist',     // Override GPU blacklist
      '--enable-gpu-rasterization', // Hardware rasterization
      '--enable-zero-copy',         // Zero-copy texture upload
    ]
  }
});

Verifying GPU acceleration

Use Helios.diagnose() to check WebGL support:

const report = await Helios.diagnose();

console.log('WebGL:', report.webgl);     // true if supported
console.log('WebGL2:', report.webgl2);   // true if supported
console.log('WebCodecs:', report.webCodecs); // true if supported

For renderer diagnostics:

const diagnostics = await renderer.diagnose();

console.log('FFmpeg:', diagnostics.ffmpeg.version);
console.log('HW Accel:', diagnostics.ffmpeg.hwaccels);
console.log('Codecs:', diagnostics.browser.codecs);

Hardware encoder selection

WebCodecs automatically selects hardware encoders when available:

const config = {
  codec: 'avc1.4d002a', // H.264 High Profile
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  framerate: 30
};

const { supported } = await VideoEncoder.isConfigSupported(config);
console.log('H.264 supported:', supported);

The renderer will fall back to software encoding if hardware acceleration is unavailable.

WebGL best practices

Canvas resolution

Match canvas resolution to export resolution:

const width = 1920;
const height = 1080;

canvas.width = width;
canvas.height = height;

renderer.setSize(width, height, false); // Three.js

Avoid using CSS to scale canvas. This wastes GPU resources rendering at high resolution, then downscaling.

Texture management

Dispose unused textures

texture.dispose();
geometry.dispose();
material.dispose();

Use compressed textures

const loader = new THREE.CompressedTextureLoader();
const texture = loader.load('texture.ktx2');

Limit texture size

const maxSize = renderer.capabilities.maxTextureSize; // Usually 4096 or 8192

Avoid per-frame allocations

// Bad: Creates garbage every frame
helios.subscribe(() => {
  const temp = new THREE.Vector3(); // Allocates memory
  cube.position.copy(temp);
});

// Good: Reuse objects
const temp = new THREE.Vector3();
helios.subscribe(() => {
  temp.set(x, y, z);
  cube.position.copy(temp);
});

Frame timing patterns

Helios provides frame-accurate timing. Use currentFrame or currentTime for animations.

Linear interpolation

helios.subscribe((state) => {
  const progress = state.currentFrame / (state.duration * state.fps);
  cube.rotation.y = progress * Math.PI * 2;
});

Keyframe-based animation

function getValueAtTime(keyframes: [number, number][], time: number) {
  for (let i = 0; i < keyframes.length - 1; i++) {
    const [t1, v1] = keyframes[i];
    const [t2, v2] = keyframes[i + 1];
    
    if (time >= t1 && time <= t2) {
      const progress = (time - t1) / (t2 - t1);
      return v1 + (v2 - v1) * progress;
    }
  }
  return keyframes[keyframes.length - 1][1];
}

const keyframes: [number, number][] = [
  [0, 0],
  [2, 5],
  [5, -3],
  [10, 0]
];

helios.subscribe((state) => {
  cube.position.x = getValueAtTime(keyframes, state.currentTime);
});

Debugging canvas renders

Visual debugging

Render in headed mode to see what’s happening:

helios render composition.html --no-headless

Export canvas for inspection

helios.subscribe((state) => {
  // Draw frame
  renderer.render(scene, camera);
  
  // Export for debugging
  if (state.currentFrame === 30) {
    canvas.toBlob((blob) => {
      const url = URL.createObjectURL(blob!);
      console.log('Frame 30:', url);
    });
  }
});

Monitor performance

const stats = {
  drawCalls: 0,
  triangles: 0
};

helios.subscribe((state) => {
  renderer.info.reset();
  renderer.render(scene, camera);
  
  stats.drawCalls = renderer.info.render.calls;
  stats.triangles = renderer.info.render.triangles;
  
  if (state.currentFrame % 30 === 0) {
    console.log('Stats:', stats);
  }
});

Next steps

Learn about custom drivers for specialized renderers
Optimize with performance tuning
Debug issues with diagnostics

Get Started

Core Concepts

Framework Guides

Features

Rendering

Tools

Component Registry

Advanced

Canvas and WebGL rendering

Overview

Automatic path detection

Canvas rendering path

How it works

Basic setup

Three.js optimization

Basic Three.js composition

Performance tips for Three.js

Pixi.js rendering

GPU acceleration setup

Browser configuration

Verifying GPU acceleration

Hardware encoder selection

WebGL best practices

Canvas resolution

Texture management

Avoid per-frame allocations

Frame timing patterns

Linear interpolation

Keyframe-based animation

Debugging canvas renders

Visual debugging

Export canvas for inspection

Monitor performance

Next steps

​Overview

​Automatic path detection

​Canvas rendering path

​How it works

​Basic setup

​Three.js optimization

​Basic Three.js composition

​Performance tips for Three.js

​Pixi.js rendering

​GPU acceleration setup

​Browser configuration

​Verifying GPU acceleration

​Hardware encoder selection

​WebGL best practices

​Canvas resolution

​Texture management

​Avoid per-frame allocations

​Frame timing patterns

​Linear interpolation

​Keyframe-based animation

​Debugging canvas renders

​Visual debugging

​Export canvas for inspection

​Monitor performance

​Next steps

Overview

Automatic path detection

Canvas rendering path

How it works

Basic setup

Three.js optimization

Basic Three.js composition

Performance tips for Three.js

Pixi.js rendering

GPU acceleration setup

Browser configuration

Verifying GPU acceleration

Hardware encoder selection

WebGL best practices

Canvas resolution

Texture management

Avoid per-frame allocations

Frame timing patterns

Linear interpolation

Keyframe-based animation

Debugging canvas renders

Visual debugging

Export canvas for inspection

Monitor performance

Next steps