Hardware Transcoding

This document describes setting up transcoding in Memories, specifically using hardware acceleration. Hardware acceleration can significantly improve transcoding performance, especially for high resolution videos. Memories supports transcoding using CPU, VA-API and NVENC.

Overview

Newer Intel processors come with a feature called QuickSync that can significantly boost transcoding performance (4-5x improvement over x264 is common). QuickSync can be used for hardware accelerated transcoding using the VA-API in ffmpeg.

Note: VA-API acceleration may also work with some AMD GPUs.

To configure VAAPI, you need to have /dev/dri available to the Nextcloud instance with the www-data in the group owning the drivers. You also need the correct drivers and a compatible version of ffmpeg installed (older versions may not work with modern hardware).

NVIDIA GPUs support hardware transcoding using NVENC.

Hardware acceleration is optional

Hardware acceleration is optional and not required for Memories to function. If you do not have hardware acceleration, Memories will use the CPU for transcoding.

Filing bugs related to transcoding

If you have issues with hardware transcoding, reach out for help. Make sure you include details about your setup such as how the transcoder is set up, the version of each component and the logs from the transcoder.

External Transcoder

Recommended Configuration

The easiest and recommended way to use hardware transcoding in a docker environment is to use an external transcoder. This setup utilizes a separate docker container that contains the hardware drivers and ffmpeg. If you cannot do this, other installation methods are also possible.

go-vod, the transcoder of Memories, comes with a pre-built Docker image based on linuxserver/ffmpeg. The docker image connects to your Nextcloud instance and pulls the go-vod binary on startup. To set up an external transcoder, follow these steps.

Use a docker-compose.yml that runs the go-vod container and mounts the Nextcloud data directories to it. You must specify NEXTCLOUD_HOST to match the name of your Nextcloud container.
```
services:
  server:
    image: nextcloud
    volumes:
      - ncdata:/var/www/html

  go-vod:
    image: radialapps/go-vod
    restart: always
    depends_on:
      - server
    environment:
      - NEXTCLOUD_HOST=https://your-nextcloud-url
      # - NEXTCLOUD_ALLOW_INSECURE=1 # (self-signed certs or no HTTPS)
      - NVIDIA_VISIBLE_DEVICES=all
    devices:
      - /dev/dri:/dev/dri # VA-API (omit for NVENC)
    volumes:
      - ncdata:/var/www/html:ro
    # runtime: nvidia # (NVENC)
```
Device and volume bindings

In this example, the VA-API devices in /dev/dri are passed to the container, along with the Nextcloud data directory (as readonly). All volumes must be mounted at the same location as the Nextcloud container.

What to set in NEXTCLOUD_HOST?

The NEXTCLOUD_HOST environment variable must be set to the URL of your Nextcloud instance. If you are using a reverse proxy, you must set this to the URL of the reverse proxy. If you are using a self-signed certificate or http, you must also set NEXTCLOUD_ALLOW_INSECURE=1. This URL is used to download the transcoder binary and to connect to the Nextcloud instance.

Setup for NVENC

If you want to use NVENC instead of VA-API, uncomment the runtime line and remove the devices section above. You will need to install the NVIDIA Container Toolkit on your host. You may also need to switch to the CUDA scaler in the Memories admin panel.
You can now configure the go-vod connect address in the Memories admin panel to point to the external container. go-vod uses port 47788 by default, so in our example the connection address would be set to go-vod:47788.
Finally, turn on enable external transcoder in the admin panel. This will initiate a test of the transcoder and show the result.

Your external transcoder should now be functional. You can check the transcoding logs by running docker compose logs -f go-vod.

Usage with Nextcloud AIO

If you are not using NVENC, you can use the memories community container. Relevant documentation can be found here, and general directions on using community containers here. AIO v7.7.0 or higher is required.

Otherwise, if you want to use NVENC with AIO, you will need to put the container into the nextcloud-aio network. Also the datadir of AIO needs to be mounted at the same place as in its Nextcloud container into the go-vod container. Usually this would be nextcloud_aio_nextcloud_data:/mnt/ncdata:ro or $NEXTCLOUD_DATADIR:/mnt/ncdata:ro.

Usage without Docker Compose

You can run a similar setup without docker-compose. Make sure that the Nextcloud and go-vod containers are in the same network and that the Nextcloud data directories are mounted at the same locations in both containers.

Internal Transcoder

Memories ships with an internal transcoder binary that you can directly use. In this case, you must install the drivers and ffmpeg on the same host as Nextcloud, and Memories will automatically handle starting and communicating with go-vod. This is also the default setup when you enable transcoding without hardware acceleration.

Advanced usage only

In most cases, it is easier to use an external transcoder when you need hardware acceleration. The internal transcoder is only suitable for CPU transcoding or if you do not use Docker.

NVENC

These instructions mostly focus on VA-API. For NVENC, you may find further useful pointers in this build script.

Bare Metal

If you are running Nextcloud on bare metal, you can install the drivers and ffmpeg directly on the host. If you are running nextcloud in a Virtual Magine or LXC container configuration, you will also need to pass through the hardware resource to the nextcloud machine. Some helpful guides can be found for Proxmox VM / LXC Container.

On the Nextcloud machine, you need to make sure that the www-data user has access to the /dev/dri devices. You can do this by adding the www-data user to the appropriate groups.

## Ubuntu
sudo apt-get update
sudo apt-get install -y intel-media-va-driver-non-free ffmpeg # install VA-API drivers
sudo usermod -aG video www-data # add www-data to the video group (may be different)

## Alpine
apk update
apk add --no-cache bash ffmpeg libva-utils libva-vdpau-driver libva-intel-driver intel-media-driver mesa-va-gallium

In some cases, along with adding www-data to the appropriate groups, you may also need to set the permissions of the device manually:

sudo chmod 666 /dev/dri/renderD128

You can run a test using a sample video file to check if VA-API is working correctly for the www-data user:

# It may be best to run the following test from within your
# Nextcloud data directory (e.g. /mnt/ncdata/<user>/files/)

# download sample or or use any other video file
wget https://github.com/pulsejet/memories-assets/raw/main/sample.mp4
chown www-data:www-data sample.mp4

# check if VA-API is working
sudo -u www-data \
  ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi \
  -i 'sample.mp4' -vcodec h264_vaapi \
  output-www-data.mp4

Beware of old ffmpeg and driver versions

Some package repositories distribute old ffmpeg versions that do not support some modern hardware. (e.g., the VA-API driver installed by apt in the current debian image used by Nextcloud only supports up to 10th generation Intel Ice Lake CPUs). To ensure you have a compatible version, you may want to remove your existing ffmpeg version and build the drivers and ffmpeg from source. This script for VA-API or this one for NVENC might be useful.

Docker

Use an external transcoder

If you need hardware transcoding and use Docker, it can be significantly easier to use an external transcoder. See above for instructions. The instructions below, as a result, are mostly historical and not recommended for normal usage.

If you use Docker and want to use the internal transcoder, you need to:

Pass the /dev/dri device to the Nextcloud container. In docker-compose.yml:

nc:
  build: .
  restart: always
  devices:
    - /dev/dri:/dev/dri

Make sure the right drivers are installed. This can be done using a custom Dockerfile as illustrated below.

FROM nextcloud:stable-fpm

RUN apt-get update && \
    apt-get install -y lsb-release && \
    echo "deb http://ftp.debian.org/debian $(lsb_release -cs) non-free" >> \
       /etc/apt/sources.list.d/intel-graphics.list && \
    apt-get update && \
    apt-get install -y intel-media-va-driver-non-free ffmpeg && \
    rm -rf /var/lib/apt/lists/*

COPY start.sh /
CMD /start.sh

The start.sh should add the user to the video group and start php-fpm.

#!/bin/bash
GID=`stat -c "%g" /dev/dri/renderD128`
groupadd -g $GID render2 || true # sometimes this is needed
GROUP=`getent group $GID | cut -d: -f1`
usermod -aG $GROUP www-data

php-fpm

If you're using the linuxserver/nextcloud image, based on Alpine, you can add the following to the docker-compose.yml file to install the VA-API drivers and use the internal transcoder directly.

devices:
  - /dev/dri:/dev/dri
environment:
  - DOCKER_MODS=linuxserver/mods:universal-package-install
  - INSTALL_PACKAGES=libva|libva-intel-driver|intel-media-driver|mesa-va-gallium

Troubleshooting

Basic Steps

If you have trouble with trancoding, try the following steps:

Check the admin panel for any errors. It may be possible that Memories cannot connect to the transcoder or you have a go-vod version mismatch.
Check the JS console and the logs of the transcoder. See below for instructions.
The admin panel lists a few options that work around driver bugs. For instance, if your portrait videos are rotated on VA-API or your NVENC stream hangs, try enabling these workarounds.
If you are using the internal transcoder, make sure you are running a new enough version of ffmpeg (shown in the admin panel). Generally you would need at least ffmpeg v5.x for most modern hardware but many operating systems ship with v4.x. One troubleshooting step is to build ffmpeg and the hardware drivers from source.
It may be helpful to run a manual test of ffmpeg in the same environment as the transcoder. See above for instructions. Note that the transcoder output / logs contain the full ffmpeg command used for each transcode.

Logging

When running an external transcoder, the logs go to the container's stdout. You can view them using

docker compose logs -f go-vod     # for docker-compose
docker logs -f <container-name>   # if not using docker-compose

When using the internal transcoder, the logs go to /tmp/go-vod/<instance-id>.log, where <instance-id is a unique ID for your Nextcloud instance that can be found in config.php. You can view them as illustrated below.

tail -f /tmp/go-vod/<instance-id>.log # bare metal

docker exec -it <container-name> cat /tmp/go-vod/<instance-id>.log # Docker