We will demonstrate the basics of the libplacebo GPU output API with a worked +example. The goal is to show a simple color on screen.
+pl_logAlmost all major entry-points into libplacebo require providing a log
+callback (or NULL to disable logging). This is abstracted into the pl_log
+object type, which we can create with
+pl_log_create:
Compiling
+You can compile this example with:
+ +The parameter PL_API_VER has no special significance and is merely included
+for historical reasons. Aside from that, this snippet introduces a number of
+core concepts of the libplacebo API:
For extensibility, almost all libplacebo calls take a pointer to a const
+struct pl_*_params, into which all extensible parameters go. For convenience,
+libplacebo provides macros which create anonymous params structs on the stack
+(and also fill in default parameters). Note that this only works for C99 and
+above, users of C89 and C++ must initialize parameter structs manually.
Under the hood, pl_log_params(...) just translates to &((struct
+pl_log_params) { /* default params */, ... }). This style of API allows
+libplacebo to effectively simulate optional named parameters.
On default parameters
+Wherever possible, parameters are designed in such a way that {0} gives
+you a minimal parameter structure, with default behavior and no optional
+features enabled. This is done for forwards compatibility - as new
+features are introduced, old struct initializers will simply opt out of
+them.
All libplacebo objects must be destroyed manually using the corresponding
+pl_*_destroy call, which takes a pointer to the variable the object is
+stored in. The resulting variable is written to NULL. This helps prevent
+use-after-free bugs.
NULL
+As a general rule, all libplacebo destructors are safe to call on
+variables containing NULL. So, users need not explicitly NULL-test
+before calling destructors on variables.
While libplacebo can work in isolation, to render images offline, for the sake +of this guide we want to provide something graphical on-screen. As such, we +need to create some sort of window. Libplacebo provides no built-in mechanism +for this, it assumes the API user will already have a windowing system +in-place.
+Complete examples (based on GLFW and SDL) can be found in the libplacebo +demos. But +for now, we will focus on getting a very simple window on-screen using GLFW:
+Compiling
+We now also need to include the glfw3 library to compile this example.
+ +pl_gpuAll GPU operations are abstracted into an internal pl_gpu object, which
+serves as the primary entry-point to any sort of GPU interaction. This object
+cannot be created directly, but must be obtained from some graphical API:
+currently there are Vulkan, OpenGL or D3D11. A pl_gpu can be accessed from
+an API-specific object like pl_vulkan, pl_opengl and pl_d3d11.
In this guide, for simplicity, we will be using OpenGL, simply because that's +what GLFW initializes by default.
+Setting this allows the resulting pl_gpu to be thread-safe, which
+ enables asynchronous transfers to be used. The alternative is to simply
+ call glfwMakeContextCurrent once after creating the window.
This method of making the context current is generally preferred, +however, so we've demonstrated it here for completeness' sake.
+All access to window-based rendering commands are abstracted into an object
+known as a "swapchain" (from Vulkan terminology), including the default
+backbuffers on D3D11 and OpenGL. If we want to present something to screen,
+we need to first create a pl_swapchain.
We can use this swapchain to perform the equivalent of gl*SwapBuffers:
We change this from glfwWaitEvents to glfwPollEvents because
+ we now want to re-run our main loop once per vsync, rather than only when
+ new events arrive. The pl_swapchain_swap_buffers call will ensure
+ that this does not execute too quickly.
The swapchain needs to be resized to fit the size of the window, which in + GLFW is handled by listening to a callback. In addition to setting this + callback, we also need to inform the swapchain of the initial window size.
+Note that the pl_swapchain_resize function handles both resize requests
+and size queries - hence, the actual swapchain size is returned back to
+the passed variables.
With a swapchain in hand, we're now equipped to start drawing pixels to the +screen:
+If pl_swapchain_start_frame fails, it typically means the window is
+ hidden, minimized or blocked. This is not a fatal condition, and as such
+ we simply want to process window events until we can resume rendering.
If pl_swapchain_submit_frame fails, it typically means the window has
+ been lost, and further rendering commands are not expected to succeed.
+ As such, in this case, we simply terminate the example program.
Our main render loop has changed into a combination of
+pl_swapchain_start_frame, rendering, and pl_swapchain_submit_frame. To
+start with, we simply use the pl_tex_clear function to blit a constant
+orange color to the framebuffer.
The previous code snippet represented our first foray into the pl_gpu API.
+For more detail on this API, see the GPU API section. But as a
+general rule of thumb, all pl_gpu-level operations are thread safe,
+asynchronous (except when returning something to the CPU), and internally
+refcounted (so you can destroy all objects as soon as you no longer need the
+reference).
In the example loop, pl_swapchain_swap_buffers is the only operation that
+actually flushes commands to the GPU. You can force an early flush with
+pl_gpu_flush() or pl_gpu_finish(), but other than that, commands will
+"queue" internally and complete asynchronously at some unknown point in time,
+until forward progress is needed (e.g. pl_tex_download).
We have demonstrated how to create a window, how to initialize the libplacebo +API, create a GPU instance based on OpenGL, and how to write a basic rendering +loop that blits a single color to the framebuffer.
+Here is a complete transcript of the example we built in this section:
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22 +23 +24 +25 +26 +27 +28 +29 +30 +31 +32 +33 +34 +35 +36 +37 +38 +39 +40 +41 +42 +43 +44 +45 +46 +47 +48 +49 +50 +51 +52 +53 +54 +55 +56 +57 +58 +59 +60 +61 +62 +63 +64 +65 +66 +67 +68 +69 +70 +71 +72 +73 +74 +75 +76 +77 +78 +79 +80 +81 +82 +83 +84 +85 +86 +87 +88 +89 +90 +91 +92 +93 +94 +95 +96 | |
libplacebo supports the same custom shader syntax used by +mpv, with some important +changes. This document will serve as a complete reference for this syntax.
+In general, user shaders are divided into distinct blocks. Each block can
+define a shader, a texture, a buffer, or a tunable parameter. Each block
+starts with a collection of header directives, which are lines starting with
+the syntax //!.
As an example, here is a simple shader that simply inverts the video signal:
+This shader defines one block - a shader block which hooks into the two
+texture stages LUMA and RGB, binds the hooked texture, inverts the value
+of the rgb channels, and then returns the modified color.
In a few contexts, shader directives accept arithmetic expressions, denoted by
+<expr> in the listing below. For historical reasons, all expressions are
+given in reverse polish notation
+(RPN), and the only
+value type is a floating point number. The following value types and
+arithmetic operations are available:
1.234: Literal float constant, evaluates to itself.NAME.w, NAME.width: Evaluates to the width of a texture with name NAME.NAME.h, NAME.height: Evaluates to the height of a texture with name NAME.PAR: Evaluates to the value of a tunable shader parameter with name PAR.+: Evaluates to X+Y.-: Evaluates to X-Y.*: Evaluates to X*Y./: Evaluates to X/Y.%: Evaluates to fmod(X, Y).>: Evaluates to (X > Y) ? 1.0 : 0.0.<: Evaluates to (X < Y) ? 1.0 : 0.0.=: Evaluates to fuzzy_eq(X, Y) ? 1.0 : 0.0, with some tolerance to
+ allow for floating point inaccuracy. (Around 1 ppm)!: Evaluates to X ? 0.0 : 1.0.Note that + and * can be used as suitable replacements for the otherwise
+absent boolean logic expressions (|| and &&).
Shaders are the default block type, and have no special syntax to indicate
+their presence. Shader stages contain raw GLSL code that will be
+(conditionally) executed. This GLSL snippet must define a single function
+vec4 hook(), or void hook() for compute shaders.
During the execution of any shader, the following global variables are made +available:
+int frame: A raw counter tracking the number of executions of this shader
+ stage.float random: A pseudo-random float uniformly distributed in the range
+ [0,1).vec2 input_size: The nominal size (in pixels) of the original input image.vec2 target_size: The nominal size (in pixels) of the output rectangle.vec2 tex_offset: The nominal offset (in pixels), of the original input crop.vec4 linearize(vec4 color): Linearize the input color according to the
+ image's tagged gamma function.vec4 delinearize(vec4 color): Opposite counterpart to linearize.Shader stages accept the following directives:
+HOOK <texture>A HOOK directive determines when a shader stage is run. During internal
+processing, libplacebo goes over a number of pre-defined hook points at set
+points in the processing pipeline. It is only possible to intercept the image,
+and run custom shaders, at these fixed hook points.
Here is a current list of hook points:
+RGB: Input plane containing RGB valuesLUMA: Input plane containing a Y valueCHROMA: Input plane containing chroma values (one or both)ALPHA: Input plane containing a single alpha valueXYZ: Input plane containing XYZ valuesCHROMA_SCALED: Chroma plane, after merging and upscaling to luma sizeALPHA_SCALED: Alpha plane, after upscaling to luma sizeNATIVE: Merged input planes, before any sort of color conversion (as-is)MAIN: After conversion to RGB, before linearization/scalingLINEAR: After conversion to linear light (for scaling purposes)SIGMOID: After conversion to sigmoidized light (for scaling purposes)PREKERNEL: Immediately before the execution of the main scaler kernelPOSTKERNEL: Immediately after the execution of the main scaler kernelSCALED: After scaling, in either linear or non-linear light RGBPREOUTPUT: After color conversion to target colorspace, before alpha blendingOUTPUT: After alpha blending, before dithering and final output passMAINPRESUB
In mpv, MAIN and MAINPRESUB are separate shader stages, because the
+mpv option --blend-subtitles=video allows rendering overlays directly
+onto the pre-scaled video stage. libplacebo does not support this feature,
+and as such, the MAINPRESUB shader stage does not exist. It is still
+valid to refer to this name in shaders, but it is handled identically to
+MAIN.
It's possible for a hook point to never fire. For example, SIGMOID will not
+fire when downscaling, as sigmoidization only happens when upscaling.
+Similarly, LUMA/CHROMA will not fire on an RGB video and vice versa.
A single shader stage may hook multiple hook points simultaneously, for
+example, to cover both LUMA and RGB cases with the same logic. (See the
+example shader in the introduction)
BIND <texture>The BIND directive makes a texture available for use in the shader. This can
+be any of the previously named hook points, a custom texture define by a
+TEXTURE block, a custom texture saved by a SAVE directive, or the special
+value HOOKED which allows binding whatever texture hook dispatched this
+shader stage.
A bound texture will define the following GLSL functions (as macros):
+sampler2D NAME_raw: A reference to the raw texture sampler itself.vec2 NAME_pos: The texel coordinates of the current pixel.vec2 NAME_map(ivec2 id): A function that maps from gl_GlobalInvocationID
+ to texel coordinates. (Compute shaders)vec2 NAME_size: The size (in pixels) of the texture.vec2 NAME_pt: Convenience macro for 1.0 / NAME_size. The size of a
+ single pixel (in texel coordinates).vec2 NAME_off: The sample offset of the texture. Basically, the pixel
+ coordinates of the top-left corner of the sampled area.float NAME_mul: The coefficient that must be multiplied into sampled
+ values in order to rescale them to [0,1].vec4 NAME_tex(vec2 pos): A wrapper around NAME_mul * textureLod(NAME_raw,
+ pos, 0.0).vec4 NAME_texOff(vec2 offset): A wrapper around NAME_tex(NAME_pos + NAME_pt * offset).
+ This can be used to easily access adjacent pixels, e.g. NAME_texOff(-1,2)
+ samples a pixel one to the left and two to the bottom of the current
+ location.vec4 NAME_gather(vec2 pos, int c): A wrapper around
+ NAME_mul * textureGather(pos, c), with appropriate scaling. (Only when
+ supported1)Rotation matrix
+For compatibility with mpv, we also define a mat2 NAME_rot which is
+simply equal to a 2x2 identity matrix. libplacebo never rotates input
+planes - all rotation happens during the final output to the display.
This same directive can also be used to bind buffer blocks (i.e.
+uniform/storage buffers), as defined by the BUFFER directive.
SAVE <texture>By default, after execution of a shader stage, the resulting output is
+captured back into the same hooked texture that triggered the shader. This
+behavior can be overridden using the explicit SAVE directive. For example,
+a shader might need access to a low-res version of the luma input texture in
+order to process chroma:
This shader binds both luma and chroma and resizes the luma plane down to the
+size of the chroma plane, saving the result as a new texture LUMA_LOWRES. In
+general, you can pick any name you want, here.
DESC <description>This purely informative directive simply gives the shader stage a name. This +is the name that will be reported to the shader stage and execution time +metrics.
+OFFSET <xo yo | ALIGN>This directive indicates a pixel shift (offset) introduced by this pass. These +pixel offsets will be accumulated and corrected automatically as part of plane +alignment / main scaling.
+A special value of ALIGN will attempt to counteract any existing offset of
+the hooked texture by aligning it with reference plane (i.e. luma). This can
+be used to e.g. introduce custom chroma scaling in a way that doesn't break
+chroma subtexel offsets.
An example:
+This (slightly silly) shader simply shifts the entire sampled region to the
+bottom right by 100.5 pixels, and propagates this shift to the main scaler
+using the OFFSET directive. As such, the end result of this is that there is
+no visible shift of the overall image, but some detail (~100 pixels) near the
+bottom-right border is lost due to falling outside the bounds of the texture.
WIDTH <expr>, HEIGHT <expr>These directives can be used to override the dimensions of the resulting
+texture. Note that not all textures can be resized this way. Currently, only
+RGB, LUMA, CHROMA, XYZ, NATIVE and MAIN are resizable. Trying to
+save a texture with an incompatible size to any other shader stage will result
+in an error.
WHEN <expr>This directive takes an expression that can be used to make shader stages +conditionally executed. If this evaluates to 0, the shader stage will be +skipped.
+Example:
+This example defines a shader stage that only conditionally executes itself
+if the value of the intensity shader parameter is non-zero.
COMPONENTS <num>This directive overrides the number of components present in a texture.
+For example, if you want to extract a one-dimensional feature map from the
+otherwise 3 or 4 dimensional MAIN texture, you can use this directive to
+save on memory bandwidth and consumption by having libplacebo only allocate a
+one-component texture to store the feature map in:
COMPUTE <bw> <bh> [<tw> <th>]This directive specifies that the shader should be treated as a compute
+shader, with the block size bw and bh. The compute shader will be
+dispatched with however many blocks are necessary to completely tile over the
+output. Within each block, there will be tw*th threads, forming a single
+work group. In other words: tw and th specify the work group size, which
+can be different from the block size. So for example, a compute shader with
+bw = bh = 32 and tw = th = 8 running on a 500x500 texture would dispatch
+16x16 blocks (rounded up), each with 8x8 threads.
Instead of defining a vec4 hook(), compute shaders must define a void
+hook() which results directly to the output texture, a writeonly image2D
+out_image made available to the shader stage.
For example, here is a shader executing a single-pass 41x41 convolution +(average blur) on the luma plane, using a compute shader to share sampling +work between adjacent threads in a work group:
+Custom textures can be defined and made available to shader stages using
+TEXTURE blocks. These can be used to provide e.g. LUTs or pre-trained
+weights.
The data for a texture is provided as a raw hexadecimal string encoding the +in-memory representation of a texture, according to its given texture format, +for example:
+Texture blocks accept the following directives:
+TEXTURE <name>This must be the first directive in a texture block, and marks it as such. The
+name given is the name that the texture will be referred to (via BIND
+directives).
SIZE <width> [<height> [<depth>]]This directive gives the size of the texture, as integers. For example,
+//!SIZE 512 512 marks a 512x512 texture block. Textures can be 1D, 2D or 3D
+depending on the number of coordinates specified.
FORMAT <fmt>This directive specifies the texture format. A complete list of known textures
+is exposed as part of the pl_gpu struct metadata, but they follow the format
+convention rgba8, rg16hf, rgba32f, r64i and so on.
FILTER <LINEAR | NEAREST>This directive specifies the texture magnification/minification filter.
+BORDER <CLAMP | REPEAT | MIRROR>This directive specifies the border clamping method of the texture.
+STORAGEIf present, this directive marks the texture as a storage image. It will still
+be initialized with the initial values, but rather than being bound as a
+read-only and immutable sampler2D, it is bound as a readwrite coherent
+image2D. Such texture scan be used to, for example, store persistent state
+across invocations of the shader.
Custom uniform / storage shader buffer blocks can be defined using BUFFER
+directives.
The (initial) data for a buffer is provided as a raw hexadecimal string +encoding the in-memory representation of a buffer in the corresponding GLSL +packing layout (std140 or std430 for uniform and storage blocks, +respectively):
+Buffer blocks accept the following directives:
+BUFFER <name>This must be the first directive in a buffer block, and marks it as such. The
+name given is mostly cosmetic, as individual variables can be accessed
+directly using the names given in the corresponding VAR directives.
STORAGEIf present, this directive marks the buffer as a (readwrite coherent) shader +storage block, instead of a readonly uniform buffer block. Such storage blocks +can be used to track and evolve state across invocations of this shader.
+Storage blocks may also be initialized with default data, but this is
+optional. They can also be initialized as part of the first shader execution
+(e.g. by testing for frame == 0).
VAR <type> <name>This directive appends a new variable to the shader block, with GLSL type
+<type> and shader name <name>. For example, VAR float foo introduces a
+float foo; member into the buffer block, and VAR mat4 transform introduces
+a mat4 transform; member.
It is also possible to introduce array variables, using [N] as part of the
+variable name.
Finally, the PARAM directive allows introducing tunable shader parameters,
+which are exposed programmatically as part of the C API (pl_hook).2
The default value of a parameter is given as the block body, for example:
+Parameters accept the following directives:
+PARAM <name>This must be the first directive in a parameter block, and marks it as such. +The name given is the name that will be used to refer to this parameter in +GLSL code.
+DESC <description>This directive can be used to provide a friendlier description of the shader +parameter, exposed as part of the C API to end users.
+MINIMUM <value>, MAXIMUM <value>Provides the minimum/maximum value bound of this parameter. If absent, no +minimum/maximum is enforced.
+TYPE [ENUM] <DEFINE | [DYNAMIC | CONSTANT] <type>>This gives the type of the parameter, which determines what type of values it
+can hold and how it will be made available to the shader. <type> must be
+a scalar GLSL numeric type, such as int, float or uint.
If a type is ENUM, it is treated as an enumeration type. To use this, type
+must either be int or DEFINE. Instead of providing a single default value,
+the param body should be a list of all possible enumeration values (as separate
+lines). These names will be made available inside the shader body (as a
+#define), as well as inside RPN expressions (e.g. WHEN). The qualifiers
+MINIMUM and MAXIMUM are ignored for ENUM parameters, with the value
+range instead being set implicitly from the list of options.
The optional qualifiers DYNAMIC or CONSTANT mark the parameter as
+dynamically changing and compile-time constant, respectively. A DYNAMIC
+variable is assumed to change frequently, and will be grouped with other
+frequently-changing input parameters. A CONSTANT parameter will be
+introduced as a compile-time constant into the shader header, which means thy
+can be used in e.g. constant expressions such as array sizes.3
Finally, the special type TYPE DEFINE marks a variable as a preprocessor
+define, which can be used inside #if preprocessor expressions. For example:
An example of an enum parameter:
+A collection of full examples can be found in the mpv user shaders +wiki, but +here is an example of a parametrized Gaussian smoothed film grain compute +shader:
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + 13 + 14 + 15 + 16 + 17 + 18 + 19 + 20 + 21 + 22 + 23 + 24 + 25 + 26 + 27 + 28 + 29 + 30 + 31 + 32 + 33 + 34 + 35 + 36 + 37 + 38 + 39 + 40 + 41 + 42 + 43 + 44 + 45 + 46 + 47 + 48 + 49 + 50 + 51 + 52 + 53 + 54 + 55 + 56 + 57 + 58 + 59 + 60 + 61 + 62 + 63 + 64 + 65 + 66 + 67 + 68 + 69 + 70 + 71 + 72 + 73 + 74 + 75 + 76 + 77 + 78 + 79 + 80 + 81 + 82 + 83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 | |
Because these are macros, their presence can be tested for using
+ #ifdef inside the GLSL preprocessor. ↩
In mpv using --vo=gpu-next, these can be set using the
+ --glsl-shader-opts option. ↩
On supported platforms, these are implemented using specialization + constants, which can be updated at run-time without requiring a full shader + recompilation. ↩
+Shaders in libplacebo are all written in GLSL, and built up incrementally, on
+demand. Generally, all shaders for each frame are generated per frame. So
+functions like pl_shader_color_map etc. are run anew for every frame. This
+makes the renderer very stateless and allows us to directly embed relevant
+constants, uniforms etc. as part of the same code that generates the actual
+GLSL shader.
To avoid this from becoming wasteful, libplacebo uses an internal string
+building abstraction
+(pl_str_builder).
+Rather than building up a string directly, a pl_str_builder is like a list of
+string building functions/callbacks to execute in order to generate the actual
+shader. Combined with an efficient pl_str_builder_hash, this allows us to
+avoid the bulk of the string templating work for already-cached shaders.
For the vast majority of libplacebo's history, the main entry-point into the
+shader building mechanism was the GLSL() macro (and
+variants), which works like a
+printf-append:
The special macro $ is a stand-in for an identifier (ident_t), which is
+the internal type used to pass references to loaded uniforms, descriptors and
+so on:
typedef unsigned short ident_t;
+#define $ "_%hx"
+#define NULL_IDENT 0u
+
+// ...
+
+ident_t sh_var_mat3(pl_shader sh, const char *name, pl_matrix3x3 val);
+#define SH_MAT3(val) sh_var_mat3(sh, "mat", val)
+In general, constants in libplacebo are divided into three categories:
+These are values that are expected to change very infrequently (or never), or
+for which we want to generate a different shader variant per value. Such values
+should be directly formatted as numbers into the shader text: %d, %f and so
+on. This is commonly used for array sizes, constants that depend only on
+hardware limits, constants that never change (but which have a friendly name,
+like PQ_C2 above), and so on.
As an example, the debanding iterations weights are hard-coded like this, +because the debanding shader is expected to change as a result of a different +number of iterations anyway:
+%d.0 here corresponds to the iteration index i, while the %f
+ corresponds to the fixed constant M_PI * 2.These are used for tunable parameters that are expected to change infrequently
+during normal playback. These constitute by far the biggest category, and most
+parameters coming from the various _params structs should be loaded like
+this.
They are loaded using the sh_const_*() functions, which generate a
+specialization constant on supported platforms, falling back to a literal
+shader #define otherwise. For anoymous parameters, you can use the
+short-hands SH_FLOAT, SH_INT etc.:
ident_t sh_const_int(pl_shader sh, const char *name, int val);
+ident_t sh_const_uint(pl_shader sh, const char *name, unsigned int val);
+ident_t sh_const_float(pl_shader sh, const char *name, float val);
+#define SH_INT(val) sh_const_int(sh, "const", val)
+#define SH_UINT(val) sh_const_uint(sh, "const", val)
+#define SH_FLOAT(val) sh_const_float(sh, "const", val)
+Here is an example of them in action:
+The advantage of this type of shader constant is that they will be
+transparently replaced by dynamic uniforms whenever
+pl_render_params.dynamic_constants is true, which allows the renderer to
+respond more instantly to changes in the parameters (e.g. as a result of a user
+dragging a slider around). During "normal" playback, they will then be
+"promoted" to actual shader constants to prevent them from taking up registers.
For anything else, e.g. variables which are expected to change very frequently,
+you can use the generic sh_var() mechanism, which sends constants either as
+elements of a uniform buffer, or directly as push constants:
ident_t sh_var_int(pl_shader sh, const char *name, int val, bool dynamic);
+ident_t sh_var_uint(pl_shader sh, const char *name, unsigned int val, bool dynamic);
+ident_t sh_var_float(pl_shader sh, const char *name, float val, bool dynamic);
+#define SH_INT_DYN(val) sh_var_int(sh, "const", val, true)
+#define SH_UINT_DYN(val) sh_var_uint(sh, "const", val, true)
+#define SH_FLOAT_DYN(val) sh_var_float(sh, "const", val, true)
+These are used primarily when a variable is expected to change very frequently, +e.g. as a result of randomness, or for constants which depend on dynamically +computed, source-dependent variables (e.g. input frame characteristics):
+Shader macros come in three main flavors, depending on where the resulting text +should be formatted:
+GLSL: Expanded in the scope of the current main function,
+ and is related to code directly processing the current pixel value.GLSLH: Printed to the 'header', before the first function, but after
+ variables, uniforms etc. This is used for global definitions, helper
+ functions, shared memory variables, and so on.GLSLF: Printed to the footer, which is always at the end of the current
+ main function, but before returning to the caller / writing to the
+ framebuffer. Used to e.g. update SSBO state in preparation for the next
+ frame.Finally, there is a fourth category GLSLP (prelude), which is currently only
+used internally to generate preambles during e.g. compute shader translation.
Starting with libplacebo v6, the internal shader system has been augmented by a +custom macro preprocessor, which is designed to ease the boilerplate of writing +shaders (and also strip redundant whitespace from generated shaders). The code +for this is found in the +tools/glsl_preproc +directory.
+In a nutshell, this allows us to embed GLSL snippets directly as #pragma GLSL
+macros (resp. #pragma GLSLH, #pragma GLSLF):
This gets transformed, by the GLSL macro preprocessor, into an optimized shader +template invocation like the following:
+To support this style of shader programming, special syntax was invented:
+Instead of being formatted with "$", %f etc. and supplied in a big list,
+printf style, GLSL macros may directly embed shader variables:
ident_t pos, tex = sh_bind(sh, texture, ..., &pos, ...);
+#pragma GLSL vec4 color = texture($tex, $pos);
+The simplest possible shader variable is just $name, which corresponds to
+any variable of type ident_t. More complicated expression are also possible:
In the expression ${float:params->noise}, the float: prefix here transforms
+the shader variable into the equivalent of SH_FLOAT() in the legacy API,
+that is, a generic float (specialization) constant. Other possible types are:
In addition to a type specifier, the optional qualifiers dynamic and const
+will modify the variable, turning it into (respectively) a dynamically loaded
+uniform (SH_FLOAT_DYN etc.), or a hard-coded shader literal (%d, %f
+etc.):
Lines beginning with @ are not included in the GLSL as-is, but instead parsed
+as macro directives, to control the code flow inside the macro expansion:
Standard-purpose conditional. Example:
+float alpha = ...;
+@if (repr.alpha == PL_ALPHA_INDEPENDENT)
+ color.a *= alpha;
+@else
+ color.rgba *= alpha;
+The condition is evaluated outside the macro (in the enclosing scope) and +the resulting boolean variable is directly passed to the template.
+An @if block can also enclose multiple lines:
@if (threshold > 0) {
+ float thresh = ${float:threshold};
+ coeff = mix(coeff, vec2(0.0), lessThan(coeff, vec2(thresh)));
+ coeff = mix(coeff, vec2(1.0), greaterThan(coeff, vec2(1.0 - thresh)));
+@}
+This can be used to generate (unrolled) loops:
+int offset = ${const int: params->kernel_width / 2};
+float sum = 0.0;
+@for (x < params->kernel_width)
+ sum += textureLodOffset($luma, $pos, 0.0, int(@sum - offset)).r;
+This introduces a local variable, @x, which expands to an integer containing
+the current loop index. Loop indices always start at 0. Valid terminating
+conditions include < and <=, and the loop stop condition is also evaluated
+as an integer.
Alternatively, this can be used to iterate over a bitmask (as commonly used for +e.g. components in a color mask):
+float weight = /* ... */;
+vec4 color = textureLod($tex, $pos, 0.0);
+@for (c : params->component_mask)
+ sum[@c] += weight * color[@c];
+Finally, to combine loops with conditionals, the special syntax @if @(cond)
+may be used to evaluate expressions inside the template loop:
@for (i < 10) {
+ float weight = /* ... */;
+ @if @(i < 5)
+ weight = -weight;
+ sum += weight * texture(...);
+@}
+In this case, the @if conditional may only reference local (loop) variables.
This corresponds fairly straightforwardly to a normal switch/case from C:
+@switch (color->transfer) {
+@case PL_COLOR_TRC_SRGB:
+ color.rgb = mix(color.rgb * 1.0/12.92,
+ pow((color.rgb + vec3(0.055)) / 1.055, vec3(2.4)),
+ lessThan(vec3(0.04045), color.rgb));
+ @break;
+@case PL_COLOR_TRC_GAMMA18:
+ color.rgb = pow(color.rgb, vec3(1.8));
+ @break;
+@case PL_COLOR_TRC_GAMMA20:
+ color.rgb = pow(color.rgb, vec3(2.0));
+ @break;
+@case PL_COLOR_TRC_GAMMA22:
+ color.rgb = pow(color.rgb, vec3(2.2));
+ @break;
+/* ... */
+@}
+The switch body is always evaluated as an unsigned int.
This document will serve as an introduction to and usage example for the +libplacebo API. This is not +intended as a full API reference, for that you should see the repository of +header +files, +which are written to be (hopefully) understandable as-is.
+libplacebo exposes large parts of its internal abstractions publicly. This +guide will take the general approach of starting as high level as possible and +diving into the details in later chapters.
+A full listing of currently available APIs and their corresponding header +files can be seen +here.
+To get started using libplacebo, you need to install it (and its development
+headers) somehow onto your system. On most distributions, this should be as
+simple as installing the corresponding libplacebo-devel package, or the
+appropriate variants.
You can see a fill list of libplacebo packages and their names on +repology.
+API versions
+This document is targeting the "v4 API" overhaul, and as such, examples +provided will generally fail to compile on libplacebo versions below v4.x.
+Alternatively, you can install it from the source code. For that, see the +build instructions located here.
+ + + + + + +This example roughly builds off the previous entry,
+and as such will not cover the basics of how to create a window, initialize a
+pl_gpu and get pixels onto the screen.
The pl_renderer set of APIs represents the highest-level interface into
+libplacebo, and is what most users who simply want to display e.g. a video
+feed on-screen will want to be using.
The basic initialization is straightforward, requiring no extra parameters:
+What makes the renderer powerful is the large number of pl_render_params it
+exposes. By default, libplacebo provides several presets to use:
Covering all of the possible options exposed by pl_render_params is
+out-of-scope of this example and would be better served by looking at the API
+documentation.
pl_frame
+is the struct libplacebo uses to group textures and their metadata together
+into a coherent unit that can be rendered using the renderer. This is not
+currently a dynamically allocated or refcounted heap object, it is merely a
+struct that can live on the stack (or anywhere else). The actual data lives in
+corresponding pl_tex objects referenced in each of the frame's planes.
Renderer state
+The pl_renderer is conceptually (almost) stateless. The only thing that
+is needed to get a different result is to change the render params, which
+can be varied freely on every call, if the user desires.
The one case where this is not entirely true is when using frame mixing
+(see below), or when using HDR peak detection. In this case, the renderer
+can be explicitly reset using pl_renderer_flush_cache.
To upload frames, the easiest methods are made available as dedicated helpers
+in
+<libplacebo/utils/upload.h>,
+and
+<libplacebo/utils/libav.h>
+(for AVFrames). In general, I recommend checking out the demo
+programs
+for a clearer illustration of how to use them in practice.
The renderer internally generates, compiles and caches a potentially large +number of shader programs, some of which can be complex. On some platforms +(notably D3D11), these can be quite costly to recompile on every program +launch.
+As such, the renderer offers a way to save/restore its internal shader cache +from some external location (managed by the API user). The use of this API is +highly recommended:
+Cache safety
+libplacebo performs only minimal validity checking on the shader cache, +and in general, cannot possibly guard against malicious alteration of such +files. Loading a cache from an untrusted source represents a remote code +execution vector.
+One of the renderer's most powerful features is its ability to compensate +for differences in framerates between the source and display by using frame +mixing to blend +adjacent frames together.
+Using this API requires presenting the renderer, at each vsync, with a
+pl_frame_mix struct, describing the current state of the vsync. In
+principle, such structs can be constructed by hand. To do this, all of the
+relevant frames (nearby the vsync timestamp) must be collected, and their
+relative distances to the vsync determined, by normalizing all PTS values such
+that the vsync represents time 0.0 (and a distance of 1.0 represents the
+nominal duration between adjacent frames). Note that timing vsyncs, and
+determining the correct vsync duration, are both left as problems for the user
+to solve.1. Here could be an example of a valid struct:
(struct pl_frame_mix) {
+ .num_frames = 6
+ .frames = (const struct pl_frame *[]) {
+ /* frame 0 */
+ /* frame 1 */
+ /* ... */
+ /* frame 5 */
+ },
+ .signatures = (uint64_t[]) {
+ 0x0, 0x1, 0x2, 0x3, 0x4, 0x5 // (1)
+ },
+ .timestamps = (float[]) {
+ -2.4, -1.4, -0.4, 0.6, 1.6, 2.6, // (2)
+ },
+ .vsync_duration = 0.4, // 24 fps video on 60 fps display
+}
+These must be unique per frame, but always refer to the same frame. For + example, this could be based on the frame's PTS, the frame's numerical ID + (in order of decoding), or some sort of hash. The details don't matter, + only that this uniquely identifies specific frames.
+Typically, for CFR sources, frame timestamps will always be separated in + this list by a distance of 1.0. In this example, the vsync falls roughly + halfway (but not quite) in between two adjacent frames (with IDs 0x2 and + 0x3).
+Frame mixing radius
+In this example, the frame mixing radius (as determined by
+pl_frame_mix_radius is 3.0, so we include all frames that fall within
+the timestamp interval of [-3, 3). In general, you should consult this
+function to determine what frames need to be included in the
+pl_frame_mix - though including more frames than needed is not an error.
Because this API is rather unwieldy and clumsy to use directly, libplacebo
+provides a helper abstraction known as pl_queue to assist in transforming
+some arbitrary source of frames (such as a video decoder) into nicely packed
+pl_frame_mix structs ready for consumption by the pl_renderer:
This queue can be interacted with through a number of mechanisms: either +pushing frames (blocking or non-blocking), or by having the queue poll frames +(via blocking or non-blocking callback) as-needed. For a full overview of the +various methods of pushing and polling frames, check the API +documentation.
+In this example, I will assume that we have a separate decoder thread pushing
+frames into the pl_queue in a blocking manner:
Now, in our render loop, we want to call pl_queue_update with appropriate
+values to retrieve the correct frame mix for each vsync:
There is a fourth status, PL_QUEUE_MORE, which is returned only if the
+ resulting frame mix is incomplete (and the timeout was reached) -
+ basically this can only happen if the queue runs dry due to frames not
+ being supplied fast enough.
In this example, since we are setting timeout to UINT64_MAX, we will
+never get this return value.
Setting this makes pl_queue_update block indefinitely until sufficiently
+ many frames have been pushed into the pl_queue from our separate
+ decoding thread.
The frame queue also vastly simplifies the process of performing
+motion-adaptive temporal deinterlacing, by automatically linking together
+adjacent fields/frames. To take advantage of this, all you need to do is set
+the appropriate field (pl_source_frame.first_frame), as well as enabling
+deinterlacing
+parameters.
However, this may change in the future, as the recent introduction of
+ the Vulkan display timing extension may result in display timing feedback
+ being added to the pl_swapchain API. That said, as of writing, this has
+ not yet happened. ↩
This document will serve as an introduction to and usage example for the libplacebo API. This is not intended as a full API reference, for that you should see the repository of header files, which are written to be (hopefully) understandable as-is.
libplacebo exposes large parts of its internal abstractions publicly. This guide will take the general approach of starting as high level as possible and diving into the details in later chapters.
A full listing of currently available APIs and their corresponding header files can be seen here.
"},{"location":"#getting-started","title":"Getting Started","text":"To get started using libplacebo, you need to install it (and its development headers) somehow onto your system. On most distributions, this should be as simple as installing the corresponding libplacebo-devel package, or the appropriate variants.
You can see a fill list of libplacebo packages and their names on repology.
API versions
This document is targeting the \"v4 API\" overhaul, and as such, examples provided will generally fail to compile on libplacebo versions below v4.x.
Alternatively, you can install it from the source code. For that, see the build instructions located here.
"},{"location":"basic-rendering/","title":"Basic windowing / output example","text":"We will demonstrate the basics of the libplacebo GPU output API with a worked example. The goal is to show a simple color on screen.
"},{"location":"basic-rendering/#creating-a-pl_log","title":"Creating apl_log","text":"Almost all major entry-points into libplacebo require providing a log callback (or NULL to disable logging). This is abstracted into the pl_log object type, which we can create with pl_log_create:
#include <libplacebo/log.h>\npl_log pllog;\nint main()\n{\npllog = pl_log_create(PL_API_VER, pl_log_params(\n.log_cb = pl_log_color,\n.log_level = PL_LOG_INFO,\n));\n// ...\npl_log_destroy(&pllog);\nreturn 0;\n}\nCompiling
You can compile this example with:
$ gcc example.c -o example `pkg-config --cflags --libs libplacebo`\nThe parameter PL_API_VER has no special significance and is merely included for historical reasons. Aside from that, this snippet introduces a number of core concepts of the libplacebo API:
For extensibility, almost all libplacebo calls take a pointer to a const struct pl_*_params, into which all extensible parameters go. For convenience, libplacebo provides macros which create anonymous params structs on the stack (and also fill in default parameters). Note that this only works for C99 and above, users of C89 and C++ must initialize parameter structs manually.
Under the hood, pl_log_params(...) just translates to &((struct pl_log_params) { /* default params */, ... }). This style of API allows libplacebo to effectively simulate optional named parameters.
On default parameters
Wherever possible, parameters are designed in such a way that {0} gives you a minimal parameter structure, with default behavior and no optional features enabled. This is done for forwards compatibility - as new features are introduced, old struct initializers will simply opt out of them.
All libplacebo objects must be destroyed manually using the corresponding pl_*_destroy call, which takes a pointer to the variable the object is stored in. The resulting variable is written to NULL. This helps prevent use-after-free bugs.
NULL
As a general rule, all libplacebo destructors are safe to call on variables containing NULL. So, users need not explicitly NULL-test before calling destructors on variables.
While libplacebo can work in isolation, to render images offline, for the sake of this guide we want to provide something graphical on-screen. As such, we need to create some sort of window. Libplacebo provides no built-in mechanism for this, it assumes the API user will already have a windowing system in-place.
Complete examples (based on GLFW and SDL) can be found in the libplacebo demos. But for now, we will focus on getting a very simple window on-screen using GLFW:
// ...\n#include <GLFW/glfw3.h>\nconst char * const title = \"libplacebo demo\";\nint width = 800;\nint height = 600;\nGLFWwindow *window;\nint main()\n{\npllog = pl_log_create(PL_API_VER, pl_log_params(\n.log_level = PL_LOG_INFO,\n));\nif (!glfwInit())\nreturn 1;\nwindow = glfwCreateWindow(width, height, title, NULL, NULL);\nif (!window)\nreturn 1;\nwhile (!glfwWindowShouldClose(window)) {\nglfwWaitEvents();\n}\nglfwDestroyWindow(window);\nglfwTerminate();\npl_log_destroy(&pllog);\nreturn 0;\n}\nCompiling
We now also need to include the glfw3 library to compile this example.
$ gcc example.c -o example `pkg-config --cflags --libs glfw3 libplacebo`\npl_gpu","text":"All GPU operations are abstracted into an internal pl_gpu object, which serves as the primary entry-point to any sort of GPU interaction. This object cannot be created directly, but must be obtained from some graphical API: currently there are Vulkan, OpenGL or D3D11. A pl_gpu can be accessed from an API-specific object like pl_vulkan, pl_opengl and pl_d3d11.
In this guide, for simplicity, we will be using OpenGL, simply because that's what GLFW initializes by default.
// ...\npl_opengl opengl;\nstatic bool make_current(void *priv);\nstatic void release_current(void *priv);\nint main()\n{\n// ...\nwindow = glfwCreateWindow(width, height, title, NULL, NULL);\nif (!window)\nreturn 1;\nopengl = pl_opengl_create(pllog, pl_opengl_params(\n.get_proc_addr = glfwGetProcAddress,\n.allow_software = true, // allow software rasterers\n.debug = true, // enable error reporting\n.make_current = make_current, // (1)\n.release_current = release_current,\n));\nif (!opengl)\nreturn 2;\nwhile (!glfwWindowShouldClose(window)) {\nglfwWaitEvents();\n}\npl_opengl_destroy(&opengl);\nglfwDestroyWindow(window);\nglfwTerminate();\npl_log_destroy(&pllog);\nreturn 0;\n}\nstatic bool make_current(void *priv)\n{\nglfwMakeContextCurrent(window);\nreturn true;\n}\nstatic void release_current(void *priv)\n{\nglfwMakeContextCurrent(NULL);\n}\nSetting this allows the resulting pl_gpu to be thread-safe, which enables asynchronous transfers to be used. The alternative is to simply call glfwMakeContextCurrent once after creating the window.
This method of making the context current is generally preferred, however, so we've demonstrated it here for completeness' sake.
All access to window-based rendering commands are abstracted into an object known as a \"swapchain\" (from Vulkan terminology), including the default backbuffers on D3D11 and OpenGL. If we want to present something to screen, we need to first create a pl_swapchain.
We can use this swapchain to perform the equivalent of gl*SwapBuffers:
// ...\npl_swapchain swchain;\nstatic void resize_cb(GLFWwindow *win, int new_w, int new_h)\n{\nwidth = new_w;\nheight = new_h;\npl_swapchain_resize(swchain, &width, &height);\n}\nint main()\n{\n// ...\nif (!opengl)\nreturn 2;\nswchain = pl_opengl_create_swapchain(opengl, pl_opengl_swapchain_params(\n.swap_buffers = (void (*)(void *)) glfwSwapBuffers,\n.priv = window,\n));\nif (!swchain)\nreturn 2;\n// (2)\nif (!pl_swapchain_resize(swchain, &width, &height))\nreturn 2;\nglfwSetFramebufferSizeCallback(window, resize_cb);\nwhile (!glfwWindowShouldClose(window)) {\npl_swapchain_swap_buffers(swchain);\nglfwPollEvents(); // (1)\n}\npl_swapchain_destroy(&swchain);\npl_opengl_destroy(&opengl);\nglfwDestroyWindow(window);\nglfwTerminate();\npl_log_destroy(&pllog);\nreturn 0;\n}\nWe change this from glfwWaitEvents to glfwPollEvents because we now want to re-run our main loop once per vsync, rather than only when new events arrive. The pl_swapchain_swap_buffers call will ensure that this does not execute too quickly.
The swapchain needs to be resized to fit the size of the window, which in GLFW is handled by listening to a callback. In addition to setting this callback, we also need to inform the swapchain of the initial window size.
Note that the pl_swapchain_resize function handles both resize requests and size queries - hence, the actual swapchain size is returned back to the passed variables.
With a swapchain in hand, we're now equipped to start drawing pixels to the screen:
// ...\nstatic void render_frame(struct pl_swapchain_frame frame)\n{\npl_gpu gpu = opengl->gpu;\npl_tex_clear(gpu, frame.fbo, (float[4]){ 1.0, 0.5, 0.0, 1.0 });\n}\nint main()\n{\n// ...\nwhile (!glfwWindowShouldClose(window)) {\nstruct pl_swapchain_frame frame;\nwhile (!pl_swapchain_start_frame(swchain, &frame))\nglfwWaitEvents(); // (1)\nrender_frame(frame);\nif (!pl_swapchain_submit_frame(swchain))\nbreak; // (2)\npl_swapchain_swap_buffers(swchain);\nglfwPollEvents();\n}\n// ...\n}\nIf pl_swapchain_start_frame fails, it typically means the window is hidden, minimized or blocked. This is not a fatal condition, and as such we simply want to process window events until we can resume rendering.
If pl_swapchain_submit_frame fails, it typically means the window has been lost, and further rendering commands are not expected to succeed. As such, in this case, we simply terminate the example program.
Our main render loop has changed into a combination of pl_swapchain_start_frame, rendering, and pl_swapchain_submit_frame. To start with, we simply use the pl_tex_clear function to blit a constant orange color to the framebuffer.
The previous code snippet represented our first foray into the pl_gpu API. For more detail on this API, see the GPU API section. But as a general rule of thumb, all pl_gpu-level operations are thread safe, asynchronous (except when returning something to the CPU), and internally refcounted (so you can destroy all objects as soon as you no longer need the reference).
In the example loop, pl_swapchain_swap_buffers is the only operation that actually flushes commands to the GPU. You can force an early flush with pl_gpu_flush() or pl_gpu_finish(), but other than that, commands will \"queue\" internally and complete asynchronously at some unknown point in time, until forward progress is needed (e.g. pl_tex_download).
We have demonstrated how to create a window, how to initialize the libplacebo API, create a GPU instance based on OpenGL, and how to write a basic rendering loop that blits a single color to the framebuffer.
Here is a complete transcript of the example we built in this section:
Basic rendering#include <GLFW/glfw3.h>\n#include <libplacebo/log.h>\n#include <libplacebo/opengl.h>\n#include <libplacebo/gpu.h>\nconst char * const title = \"libplacebo demo\";\nint width = 800;\nint height = 600;\nGLFWwindow *window;\npl_log pllog;\npl_opengl opengl;\npl_swapchain swchain;\nstatic bool make_current(void *priv);\nstatic void release_current(void *priv);\nstatic void resize_cb(GLFWwindow *win, int new_w, int new_h)\n{\nwidth = new_w;\nheight = new_h;\npl_swapchain_resize(swchain, &width, &height);\n}\nstatic void render_frame(struct pl_swapchain_frame frame)\n{\npl_gpu gpu = opengl->gpu;\npl_tex_clear(gpu, frame.fbo, (float[4]){ 1.0, 0.5, 0.0, 1.0 });\n}\nint main()\n{\npllog = pl_log_create(PL_API_VER, pl_log_params(\n.log_cb = pl_log_color,\n.log_level = PL_LOG_INFO,\n));\nif (!glfwInit())\nreturn 1;\nwindow = glfwCreateWindow(width, height, title, NULL, NULL);\nif (!window)\nreturn 1;\nopengl = pl_opengl_create(pllog, pl_opengl_params(\n.get_proc_addr = glfwGetProcAddress,\n.allow_software = true, // allow software rasterers\n.debug = true, // enable error reporting\n.make_current = make_current,\n.release_current = release_current,\n));\nswchain = pl_opengl_create_swapchain(opengl, pl_opengl_swapchain_params(\n.swap_buffers = (void (*)(void *)) glfwSwapBuffers,\n.priv = window,\n));\nif (!swchain)\nreturn 2;\nif (!pl_swapchain_resize(swchain, &width, &height))\nreturn 2;\nglfwSetFramebufferSizeCallback(window, resize_cb);\nwhile (!glfwWindowShouldClose(window)) {\nstruct pl_swapchain_frame frame;\nwhile (!pl_swapchain_start_frame(swchain, &frame))\nglfwWaitEvents();\nrender_frame(frame);\nif (!pl_swapchain_submit_frame(swchain))\nbreak;\npl_swapchain_swap_buffers(swchain);\nglfwPollEvents();\n}\npl_swapchain_destroy(&swchain);\npl_opengl_destroy(&opengl);\nglfwDestroyWindow(window);\nglfwTerminate();\npl_log_destroy(&pllog);\nreturn 0;\n}\nstatic bool make_current(void *priv)\n{\nglfwMakeContextCurrent(window);\nreturn true;\n}\nstatic void release_current(void *priv)\n{\nglfwMakeContextCurrent(NULL);\n}\nlibplacebo supports the same custom shader syntax used by mpv, with some important changes. This document will serve as a complete reference for this syntax.
"},{"location":"custom-shaders/#overview","title":"Overview","text":"In general, user shaders are divided into distinct blocks. Each block can define a shader, a texture, a buffer, or a tunable parameter. Each block starts with a collection of header directives, which are lines starting with the syntax //!.
As an example, here is a simple shader that simply inverts the video signal:
//!HOOK LUMA\n//!HOOK RGB\n//!BIND HOOKED\nvec4 hook()\n{\nvec4 color = HOOKED_texOff(0);\ncolor.rgb = vec3(1.0) - color.rgb;\nreturn color;\n}\nThis shader defines one block - a shader block which hooks into the two texture stages LUMA and RGB, binds the hooked texture, inverts the value of the rgb channels, and then returns the modified color.
In a few contexts, shader directives accept arithmetic expressions, denoted by <expr> in the listing below. For historical reasons, all expressions are given in reverse polish notation (RPN), and the only value type is a floating point number. The following value types and arithmetic operations are available:
1.234: Literal float constant, evaluates to itself.NAME.w, NAME.width: Evaluates to the width of a texture with name NAME.NAME.h, NAME.height: Evaluates to the height of a texture with name NAME.PAR: Evaluates to the value of a tunable shader parameter with name PAR.+: Evaluates to X+Y.-: Evaluates to X-Y.*: Evaluates to X*Y./: Evaluates to X/Y.%: Evaluates to fmod(X, Y).>: Evaluates to (X > Y) ? 1.0 : 0.0.<: Evaluates to (X < Y) ? 1.0 : 0.0.=: Evaluates to fuzzy_eq(X, Y) ? 1.0 : 0.0, with some tolerance to allow for floating point inaccuracy. (Around 1 ppm)!: Evaluates to X ? 0.0 : 1.0.Note that + and * can be used as suitable replacements for the otherwise absent boolean logic expressions (|| and &&).
Shaders are the default block type, and have no special syntax to indicate their presence. Shader stages contain raw GLSL code that will be (conditionally) executed. This GLSL snippet must define a single function vec4 hook(), or void hook() for compute shaders.
During the execution of any shader, the following global variables are made available:
int frame: A raw counter tracking the number of executions of this shader stage.float random: A pseudo-random float uniformly distributed in the range [0,1).vec2 input_size: The nominal size (in pixels) of the original input image.vec2 target_size: The nominal size (in pixels) of the output rectangle.vec2 tex_offset: The nominal offset (in pixels), of the original input crop.vec4 linearize(vec4 color): Linearize the input color according to the image's tagged gamma function.vec4 delinearize(vec4 color): Opposite counterpart to linearize.Shader stages accept the following directives:
"},{"location":"custom-shaders/#hook-texture","title":"HOOK <texture>","text":"A HOOK directive determines when a shader stage is run. During internal processing, libplacebo goes over a number of pre-defined hook points at set points in the processing pipeline. It is only possible to intercept the image, and run custom shaders, at these fixed hook points.
Here is a current list of hook points:
RGB: Input plane containing RGB valuesLUMA: Input plane containing a Y valueCHROMA: Input plane containing chroma values (one or both)ALPHA: Input plane containing a single alpha valueXYZ: Input plane containing XYZ valuesCHROMA_SCALED: Chroma plane, after merging and upscaling to luma sizeALPHA_SCALED: Alpha plane, after upscaling to luma sizeNATIVE: Merged input planes, before any sort of color conversion (as-is)MAIN: After conversion to RGB, before linearization/scalingLINEAR: After conversion to linear light (for scaling purposes)SIGMOID: After conversion to sigmoidized light (for scaling purposes)PREKERNEL: Immediately before the execution of the main scaler kernelPOSTKERNEL: Immediately after the execution of the main scaler kernelSCALED: After scaling, in either linear or non-linear light RGBPREOUTPUT: After color conversion to target colorspace, before alpha blendingOUTPUT: After alpha blending, before dithering and final output passMAINPRESUB
In mpv, MAIN and MAINPRESUB are separate shader stages, because the mpv option --blend-subtitles=video allows rendering overlays directly onto the pre-scaled video stage. libplacebo does not support this feature, and as such, the MAINPRESUB shader stage does not exist. It is still valid to refer to this name in shaders, but it is handled identically to MAIN.
It's possible for a hook point to never fire. For example, SIGMOID will not fire when downscaling, as sigmoidization only happens when upscaling. Similarly, LUMA/CHROMA will not fire on an RGB video and vice versa.
A single shader stage may hook multiple hook points simultaneously, for example, to cover both LUMA and RGB cases with the same logic. (See the example shader in the introduction)
BIND <texture>","text":"The BIND directive makes a texture available for use in the shader. This can be any of the previously named hook points, a custom texture define by a TEXTURE block, a custom texture saved by a SAVE directive, or the special value HOOKED which allows binding whatever texture hook dispatched this shader stage.
A bound texture will define the following GLSL functions (as macros):
sampler2D NAME_raw: A reference to the raw texture sampler itself.vec2 NAME_pos: The texel coordinates of the current pixel.vec2 NAME_map(ivec2 id): A function that maps from gl_GlobalInvocationID to texel coordinates. (Compute shaders)vec2 NAME_size: The size (in pixels) of the texture.vec2 NAME_pt: Convenience macro for 1.0 / NAME_size. The size of a single pixel (in texel coordinates).vec2 NAME_off: The sample offset of the texture. Basically, the pixel coordinates of the top-left corner of the sampled area.float NAME_mul: The coefficient that must be multiplied into sampled values in order to rescale them to [0,1].vec4 NAME_tex(vec2 pos): A wrapper around NAME_mul * textureLod(NAME_raw, pos, 0.0).vec4 NAME_texOff(vec2 offset): A wrapper around NAME_tex(NAME_pos + NAME_pt * offset). This can be used to easily access adjacent pixels, e.g. NAME_texOff(-1,2) samples a pixel one to the left and two to the bottom of the current location.vec4 NAME_gather(vec2 pos, int c): A wrapper around NAME_mul * textureGather(pos, c), with appropriate scaling. (Only when supported1)Rotation matrix
For compatibility with mpv, we also define a mat2 NAME_rot which is simply equal to a 2x2 identity matrix. libplacebo never rotates input planes - all rotation happens during the final output to the display.
This same directive can also be used to bind buffer blocks (i.e. uniform/storage buffers), as defined by the BUFFER directive.
SAVE <texture>","text":"By default, after execution of a shader stage, the resulting output is captured back into the same hooked texture that triggered the shader. This behavior can be overridden using the explicit SAVE directive. For example, a shader might need access to a low-res version of the luma input texture in order to process chroma:
//!HOOK CHROMA\n//!BIND CHROMA\n//!BIND LUMA\n//!SAVE LUMA_LOWRES\n//!WIDTH CHROMA.w\n//!HEIGHT CHROMA.h\nvec4 hook()\n{\nreturn LUMA_texOff(0);\n}\nThis shader binds both luma and chroma and resizes the luma plane down to the size of the chroma plane, saving the result as a new texture LUMA_LOWRES. In general, you can pick any name you want, here.
DESC <description>","text":"This purely informative directive simply gives the shader stage a name. This is the name that will be reported to the shader stage and execution time metrics.
"},{"location":"custom-shaders/#offset-xo-yo-align","title":"OFFSET <xo yo | ALIGN>","text":"This directive indicates a pixel shift (offset) introduced by this pass. These pixel offsets will be accumulated and corrected automatically as part of plane alignment / main scaling.
A special value of ALIGN will attempt to counteract any existing offset of the hooked texture by aligning it with reference plane (i.e. luma). This can be used to e.g. introduce custom chroma scaling in a way that doesn't break chroma subtexel offsets.
An example:
//!HOOK LUMA\n//!BIND HOOKED\n//!OFFSET 100.5 100.5\nvec4 hook()\n{\n// Constant offset by N pixels towards the bottom right\nreturn HOOKED_texOff(-vec2(100.5));\n}\nThis (slightly silly) shader simply shifts the entire sampled region to the bottom right by 100.5 pixels, and propagates this shift to the main scaler using the OFFSET directive. As such, the end result of this is that there is no visible shift of the overall image, but some detail (~100 pixels) near the bottom-right border is lost due to falling outside the bounds of the texture.
WIDTH <expr>, HEIGHT <expr>","text":"These directives can be used to override the dimensions of the resulting texture. Note that not all textures can be resized this way. Currently, only RGB, LUMA, CHROMA, XYZ, NATIVE and MAIN are resizable. Trying to save a texture with an incompatible size to any other shader stage will result in an error.
WHEN <expr>","text":"This directive takes an expression that can be used to make shader stages conditionally executed. If this evaluates to 0, the shader stage will be skipped.
Example:
//!PARAM strength\n//!TYPE float\n//!MINIMUM 0\n1.0\n//!HOOK MAIN\n//!BIND HOOKED\n//!WHEN intensity 0 >\n//!DESC do something based on 'intensity'\n...\nThis example defines a shader stage that only conditionally executes itself if the value of the intensity shader parameter is non-zero.
COMPONENTS <num>","text":"This directive overrides the number of components present in a texture. For example, if you want to extract a one-dimensional feature map from the otherwise 3 or 4 dimensional MAIN texture, you can use this directive to save on memory bandwidth and consumption by having libplacebo only allocate a one-component texture to store the feature map in:
//!HOOK MAIN\n//!BIND HOOKED\n//!SAVE featuremap\n//!COMPONENTS 1\nCOMPUTE <bw> <bh> [<tw> <th>]","text":"This directive specifies that the shader should be treated as a compute shader, with the block size bw and bh. The compute shader will be dispatched with however many blocks are necessary to completely tile over the output. Within each block, there will be tw*th threads, forming a single work group. In other words: tw and th specify the work group size, which can be different from the block size. So for example, a compute shader with bw = bh = 32 and tw = th = 8 running on a 500x500 texture would dispatch 16x16 blocks (rounded up), each with 8x8 threads.
Instead of defining a vec4 hook(), compute shaders must define a void hook() which results directly to the output texture, a writeonly image2D out_image made available to the shader stage.
For example, here is a shader executing a single-pass 41x41 convolution (average blur) on the luma plane, using a compute shader to share sampling work between adjacent threads in a work group:
//!HOOK LUMA\n//!BIND HOOKED\n//!COMPUTE 32 32\n//!DESC avg convolution\n// Kernel size, 41x41 as an example\nconst ivec2 ksize = ivec2(41, 41);\nconst ivec2 offset = ksize / 2;\n// We need to load extra source texels to account for padding due to kernel\n// overhang\nconst ivec2 isize = ivec2(gl_WorkGroupSize) + ksize - 1;\nshared float inp[isize.y][isize.x];\nvoid hook()\n{\n// load texels into shmem\nivec2 base = ivec2(gl_WorkGroupID) * ivec2(gl_WorkGroupSize);\nfor (uint y = gl_LocalInvocationID.y; y < isize.y; y += gl_WorkGroupSize.y) {\nfor (uint x = gl_LocalInvocationID.x; x < isize.x; x += gl_WorkGroupSize.x)\ninp[y][x] = texelFetch(HOOKED_raw, base + ivec2(x,y) - offset, 0).x;\n}\n// synchronize threads\nbarrier();\n// do convolution\nfloat sum;\nfor (uint y = 0; y < ksize.y; y++) {\nfor (uint x = 0; x < ksize.x; x++)\nsum += inp[gl_LocalInvocationID.y+y][gl_LocalInvocationID.x+x];\n}\nvec4 color = vec4(HOOKED_mul * sum / (ksize.x * ksize.y), 0, 0, 1);\nimageStore(out_image, ivec2(gl_GlobalInvocationID), color);\n}\nCustom textures can be defined and made available to shader stages using TEXTURE blocks. These can be used to provide e.g. LUTs or pre-trained weights.
The data for a texture is provided as a raw hexadecimal string encoding the in-memory representation of a texture, according to its given texture format, for example:
//!TEXTURE COLORS\n//!SIZE 3 3\n//!FORMAT rgba32f\n//!FILTER NEAREST\n//!BORDER REPEAT\n0000803f000000000000000000000000000000000000803f00000000000000000000000\n0000000000000803f00000000000000000000803f0000803f000000000000803f000000\n000000803f000000000000803f0000803f00000000000000009a99993e9a99993e9a999\n93e000000009a99193F9A99193f9a99193f000000000000803f0000803f0000803f0000\n0000\nTexture blocks accept the following directives:
"},{"location":"custom-shaders/#texture-name","title":"TEXTURE <name>","text":"This must be the first directive in a texture block, and marks it as such. The name given is the name that the texture will be referred to (via BIND directives).
SIZE <width> [<height> [<depth>]]","text":"This directive gives the size of the texture, as integers. For example, //!SIZE 512 512 marks a 512x512 texture block. Textures can be 1D, 2D or 3D depending on the number of coordinates specified.
FORMAT <fmt>","text":"This directive specifies the texture format. A complete list of known textures is exposed as part of the pl_gpu struct metadata, but they follow the format convention rgba8, rg16hf, rgba32f, r64i and so on.
FILTER <LINEAR | NEAREST>","text":"This directive specifies the texture magnification/minification filter.
"},{"location":"custom-shaders/#border-clamp-repeat-mirror","title":"BORDER <CLAMP | REPEAT | MIRROR>","text":"This directive specifies the border clamping method of the texture.
"},{"location":"custom-shaders/#storage","title":"STORAGE","text":"If present, this directive marks the texture as a storage image. It will still be initialized with the initial values, but rather than being bound as a read-only and immutable sampler2D, it is bound as a readwrite coherent image2D. Such texture scan be used to, for example, store persistent state across invocations of the shader.
Custom uniform / storage shader buffer blocks can be defined using BUFFER directives.
The (initial) data for a buffer is provided as a raw hexadecimal string encoding the in-memory representation of a buffer in the corresponding GLSL packing layout (std140 or std430 for uniform and storage blocks, respectively):
//!BUFFER buf_uniform\n//!VAR float foo\n//!VAR float bar\n0000000000000000\n//!BUFFER buf_storage\n//!VAR vec2 bat\n//!VAR int big[32];\n//!STORAGE\nBuffer blocks accept the following directives:
"},{"location":"custom-shaders/#buffer-name","title":"BUFFER <name>","text":"This must be the first directive in a buffer block, and marks it as such. The name given is mostly cosmetic, as individual variables can be accessed directly using the names given in the corresponding VAR directives.
STORAGE","text":"If present, this directive marks the buffer as a (readwrite coherent) shader storage block, instead of a readonly uniform buffer block. Such storage blocks can be used to track and evolve state across invocations of this shader.
Storage blocks may also be initialized with default data, but this is optional. They can also be initialized as part of the first shader execution (e.g. by testing for frame == 0).
VAR <type> <name>","text":"This directive appends a new variable to the shader block, with GLSL type <type> and shader name <name>. For example, VAR float foo introduces a float foo; member into the buffer block, and VAR mat4 transform introduces a mat4 transform; member.
It is also possible to introduce array variables, using [N] as part of the variable name.
Finally, the PARAM directive allows introducing tunable shader parameters, which are exposed programmatically as part of the C API (pl_hook).2
The default value of a parameter is given as the block body, for example:
//!PARAM contrast\n//!DESC Gain to apply to image brightness\n//!TYPE float\n//!MINIMUM 0.0\n//!MAXIMUM 100.0\n1.0\nParameters accept the following directives:
"},{"location":"custom-shaders/#param-name","title":"PARAM <name>","text":"This must be the first directive in a parameter block, and marks it as such. The name given is the name that will be used to refer to this parameter in GLSL code.
"},{"location":"custom-shaders/#desc-description_1","title":"DESC <description>","text":"This directive can be used to provide a friendlier description of the shader parameter, exposed as part of the C API to end users.
"},{"location":"custom-shaders/#minimum-value-maximum-value","title":"MINIMUM <value>, MAXIMUM <value>","text":"Provides the minimum/maximum value bound of this parameter. If absent, no minimum/maximum is enforced.
"},{"location":"custom-shaders/#type-enum-define-dynamic-constant-type","title":"TYPE [ENUM] <DEFINE | [DYNAMIC | CONSTANT] <type>>","text":"This gives the type of the parameter, which determines what type of values it can hold and how it will be made available to the shader. <type> must be a scalar GLSL numeric type, such as int, float or uint.
If a type is ENUM, it is treated as an enumeration type. To use this, type must either be int or DEFINE. Instead of providing a single default value, the param body should be a list of all possible enumeration values (as separate lines). These names will be made available inside the shader body (as a #define), as well as inside RPN expressions (e.g. WHEN). The qualifiers MINIMUM and MAXIMUM are ignored for ENUM parameters, with the value range instead being set implicitly from the list of options.
The optional qualifiers DYNAMIC or CONSTANT mark the parameter as dynamically changing and compile-time constant, respectively. A DYNAMIC variable is assumed to change frequently, and will be grouped with other frequently-changing input parameters. A CONSTANT parameter will be introduced as a compile-time constant into the shader header, which means thy can be used in e.g. constant expressions such as array sizes.3
Finally, the special type TYPE DEFINE marks a variable as a preprocessor define, which can be used inside #if preprocessor expressions. For example:
//!PARAM taps\n//!DESC Smoothing taps\n//!TYPE DEFINE\n//!MINIMUM 0\n//!MAXIMUM 5\n2\n//!HOOK LUMA\n//!BIND HOOKED\nconst uint row_size = 2 * taps + 1;\nconst float weights[row_size] = {\n#if taps == 0\n1.0,\n#endif\n#if taps == 1\n0.10650697891920,\n0.78698604216159,\n0.10650697891920,\n#endif\n#if taps == 2\n0.05448868454964,\n0.24420134200323,\n0.40261994689424,\n0.24420134200323,\n0.05448868454964,\n#endif\n// ...\n};\nAn example of an enum parameter:
//!PARAM csp\n//!DESC Colorspace\n//!TYPE ENUM int\nBT709\nBT2020\nDCIP3\n//!HOOK MAIN\n//!BIND HOOKED\nconst mat3 matrices[3] = {\nmat3(...), // BT709\nmat3(...), // BT2020\nmat3(...), // DCIP3\n};\n#define MAT matrices[csp]\n// ...\nA collection of full examples can be found in the mpv user shaders wiki, but here is an example of a parametrized Gaussian smoothed film grain compute shader:
//!PARAM intensity\n//!DESC Film grain intensity\n//!TYPE float\n//!MINIMUM 0\n0.1\n//!PARAM taps\n//!DESC Film grain smoothing taps\n//!TYPE DEFINE\n//!MINIMUM 0\n//!MAXIMUM 5\n2\n//!HOOK LUMA\n//!BIND HOOKED\n//!DESC Apply gaussian smoothed film grain\n//!WHEN intensity 0 >\n//!COMPUTE 32 32\nconst uint row_size = 2 * taps + 1;\nconst float weights[row_size] = {\n#if taps == 0\n1.0,\n#endif\n#if taps == 1\n0.10650697891920,\n0.78698604216159,\n0.10650697891920,\n#endif\n#if taps == 2\n0.05448868454964,\n0.24420134200323,\n0.40261994689424,\n0.24420134200323,\n0.05448868454964,\n#endif\n#if taps == 3\n0.03663284536919,\n0.11128075847888,\n0.21674532140370,\n0.27068214949642,\n0.21674532140370,\n0.11128075847888,\n0.03663284536919,\n#endif\n#if taps == 4\n0.02763055063889,\n0.06628224528636,\n0.12383153680577,\n0.18017382291138,\n0.20416368871516,\n0.18017382291138,\n0.12383153680577,\n0.06628224528636,\n0.02763055063889,\n#endif\n#if taps == 5\n0.02219054849244,\n0.04558899978527,\n0.07981140824009,\n0.11906462996609,\n0.15136080967773,\n0.16396720767670,\n0.15136080967773,\n0.11906462996609,\n0.07981140824009,\n0.04558899978527,\n0.02219054849244,\n#endif\n};\nconst uvec2 isize = uvec2(gl_WorkGroupSize) + uvec2(2 * taps);\nshared float grain[isize.y][isize.x];\n// PRNG\nfloat permute(float x)\n{\nx = (34.0 * x + 1.0) * x;\nreturn fract(x * 1.0/289.0) * 289.0;\n}\nfloat seed(uvec2 pos)\n{\nconst float phi = 1.61803398874989;\nvec3 m = vec3(fract(phi * vec2(pos)), random) + vec3(1.0);\nreturn permute(permute(m.x) + m.y) + m.z;\n}\nfloat rand(inout float state)\n{\nstate = permute(state);\nreturn fract(state * 1.0/41.0);\n}\n// Turns uniform white noise into gaussian white noise by passing it\n// through an approximation of the gaussian quantile function\nfloat rand_gaussian(inout float state) {\nconst float a0 = 0.151015505647689;\nconst float a1 = -0.5303572634357367;\nconst float a2 = 1.365020122861334;\nconst float b0 = 0.132089632343748;\nconst float b1 = -0.7607324991323768;\nfloat p = 0.95 * rand(state) + 0.025;\nfloat q = p - 0.5;\nfloat r = q * q;\nfloat g = q * (a2 + (a1 * r + a0) / (r*r + b1*r + b0));\ng *= 0.255121822830526; // normalize to [-1,1)\nreturn g;\n}\nvoid hook()\n{\n// generate grain in `grain`\nuint num_threads = gl_WorkGroupSize.x * gl_WorkGroupSize.y;\nfor (uint i = gl_LocalInvocationIndex; i < isize.y * isize.x; i += num_threads) {\nuvec2 pos = uvec2(i % isize.y, i / isize.y);\nfloat state = seed(gl_WorkGroupID.xy * gl_WorkGroupSize.xy + pos);\ngrain[pos.y][pos.x] = rand_gaussian(state);\n}\n// make writes visible\nbarrier();\n// convolve horizontally\nfor (uint y = gl_LocalInvocationID.y; y < isize.y; y += gl_WorkGroupSize.y) {\nfloat hsum = 0;\nfor (uint x = 0; x < row_size; x++) {\nfloat g = grain[y][gl_LocalInvocationID.x + x];\nhsum += weights[x] * g;\n}\n// update grain LUT\ngrain[y][gl_LocalInvocationID.x + taps] = hsum;\n}\nbarrier();\n// convolve vertically\nfloat vsum = 0.0;\nfor (uint y = 0; y < row_size; y++) {\nfloat g = grain[gl_LocalInvocationID.y + y][gl_LocalInvocationID.x + taps];\nvsum += weights[y] * g;\n}\nvec4 color = HOOKED_tex(HOOKED_pos);\ncolor.rgb += vec3(intensity * vsum);\nimageStore(out_image, ivec2(gl_GlobalInvocationID), color);\n}\nBecause these are macros, their presence can be tested for using #ifdef inside the GLSL preprocessor.\u00a0\u21a9
In mpv using --vo=gpu-next, these can be set using the --glsl-shader-opts option.\u00a0\u21a9
On supported platforms, these are implemented using specialization constants, which can be updated at run-time without requiring a full shader recompilation.\u00a0\u21a9
Shaders in libplacebo are all written in GLSL, and built up incrementally, on demand. Generally, all shaders for each frame are generated per frame. So functions like pl_shader_color_map etc. are run anew for every frame. This makes the renderer very stateless and allows us to directly embed relevant constants, uniforms etc. as part of the same code that generates the actual GLSL shader.
To avoid this from becoming wasteful, libplacebo uses an internal string building abstraction (pl_str_builder). Rather than building up a string directly, a pl_str_builder is like a list of string building functions/callbacks to execute in order to generate the actual shader. Combined with an efficient pl_str_builder_hash, this allows us to avoid the bulk of the string templating work for already-cached shaders.
For the vast majority of libplacebo's history, the main entry-point into the shader building mechanism was the GLSL() macro (and variants), which works like a printf-append:
void pl_shader_extract_features(pl_shader sh, struct pl_color_space csp)\n{\nif (!sh_require(sh, PL_SHADER_SIG_COLOR, 0, 0))\nreturn;\nsh_describe(sh, \"feature extraction\");\npl_shader_linearize(sh, &csp);\nGLSL(\"// pl_shader_extract_features \\n\"\n\"{ \\n\"\n\"vec3 lms = %f * \"$\" * color.rgb; \\n\"\n\"lms = pow(max(lms, 0.0), vec3(%f)); \\n\"\n\"lms = (vec3(%f) + %f * lms) \\n\"\n\" / (vec3(1.0) + %f * lms); \\n\"\n\"lms = pow(lms, vec3(%f)); \\n\"\n\"float I = dot(vec3(%f, %f, %f), lms); \\n\"\n\"color = vec4(I, 0.0, 0.0, 1.0); \\n\"\n\"} \\n\",\nPL_COLOR_SDR_WHITE / 10000,\nSH_MAT3(pl_ipt_rgb2lms(pl_raw_primaries_get(csp.primaries))),\nPQ_M1, PQ_C1, PQ_C2, PQ_C3, PQ_M2,\npl_ipt_lms2ipt.m[0][0], pl_ipt_lms2ipt.m[0][1], pl_ipt_lms2ipt.m[0][2]);\n}\nThe special macro $ is a stand-in for an identifier (ident_t), which is the internal type used to pass references to loaded uniforms, descriptors and so on:
typedef unsigned short ident_t;\n#define $ \"_%hx\"\n#define NULL_IDENT 0u\n// ...\nident_t sh_var_mat3(pl_shader sh, const char *name, pl_matrix3x3 val);\n#define SH_MAT3(val) sh_var_mat3(sh, \"mat\", val)\nIn general, constants in libplacebo are divided into three categories:
"},{"location":"glsl/#literal-shader-constants","title":"Literal shader constants","text":"These are values that are expected to change very infrequently (or never), or for which we want to generate a different shader variant per value. Such values should be directly formatted as numbers into the shader text: %d, %f and so on. This is commonly used for array sizes, constants that depend only on hardware limits, constants that never change (but which have a friendly name, like PQ_C2 above), and so on.
As an example, the debanding iterations weights are hard-coded like this, because the debanding shader is expected to change as a result of a different number of iterations anyway:
// For each iteration, compute the average at a given distance and\n// pick it instead of the color if the difference is below the threshold.\nfor (int i = 1; i <= params->iterations; i++) {\nGLSL(// Compute a random angle and distance\n\"d = \"$\".xy * vec2(%d.0 * \"$\", %f); \\n\" // (1)\n\"d = d.x * vec2(cos(d.y), sin(d.y)); \\n\"\n// Sample at quarter-turn intervals around the source pixel\n\"avg = T(0.0); \\n\"\n\"avg += GET(+d.x, +d.y); \\n\"\n\"avg += GET(-d.x, +d.y); \\n\"\n\"avg += GET(-d.x, -d.y); \\n\"\n\"avg += GET(+d.x, -d.y); \\n\"\n\"avg *= 0.25; \\n\"\n// Compare the (normalized) average against the pixel\n\"diff = abs(res - avg); \\n\"\n\"bound = T(\"$\" / %d.0); \\n\",\nprng, i, radius, M_PI * 2,\nthreshold, i);\nif (num_comps > 1) {\nGLSL(\"res = mix(avg, res, greaterThan(diff, bound)); \\n\");\n} else {\nGLSL(\"res = mix(avg, res, diff > bound); \\n\");\n}\n}\n%d.0 here corresponds to the iteration index i, while the %f corresponds to the fixed constant M_PI * 2.These are used for tunable parameters that are expected to change infrequently during normal playback. These constitute by far the biggest category, and most parameters coming from the various _params structs should be loaded like this.
They are loaded using the sh_const_*() functions, which generate a specialization constant on supported platforms, falling back to a literal shader #define otherwise. For anoymous parameters, you can use the short-hands SH_FLOAT, SH_INT etc.:
ident_t sh_const_int(pl_shader sh, const char *name, int val);\nident_t sh_const_uint(pl_shader sh, const char *name, unsigned int val);\nident_t sh_const_float(pl_shader sh, const char *name, float val);\n#define SH_INT(val) sh_const_int(sh, \"const\", val)\n#define SH_UINT(val) sh_const_uint(sh, \"const\", val)\n#define SH_FLOAT(val) sh_const_float(sh, \"const\", val)\nHere is an example of them in action:
void pl_shader_sigmoidize(pl_shader sh, const struct pl_sigmoid_params *params)\n{\nif (!sh_require(sh, PL_SHADER_SIG_COLOR, 0, 0))\nreturn;\nparams = PL_DEF(params, &pl_sigmoid_default_params);\nfloat center = PL_DEF(params->center, 0.75);\nfloat slope = PL_DEF(params->slope, 6.5);\n// This function needs to go through (0,0) and (1,1), so we compute the\n// values at 1 and 0, and then scale/shift them, respectively.\nfloat offset = 1.0 / (1 + expf(slope * center));\nfloat scale = 1.0 / (1 + expf(slope * (center - 1))) - offset;\nGLSL(\"// pl_shader_sigmoidize \\n\"\n\"color = clamp(color, 0.0, 1.0); \\n\"\n\"color = vec4(\"$\") - vec4(\"$\") * \\n\"\n\" log(vec4(1.0) / (color * vec4(\"$\") + vec4(\"$\")) \\n\"\n\" - vec4(1.0)); \\n\",\nSH_FLOAT(center), SH_FLOAT(1.0 / slope),\nSH_FLOAT(scale), SH_FLOAT(offset));\n}\nThe advantage of this type of shader constant is that they will be transparently replaced by dynamic uniforms whenever pl_render_params.dynamic_constants is true, which allows the renderer to respond more instantly to changes in the parameters (e.g. as a result of a user dragging a slider around). During \"normal\" playback, they will then be \"promoted\" to actual shader constants to prevent them from taking up registers.
For anything else, e.g. variables which are expected to change very frequently, you can use the generic sh_var() mechanism, which sends constants either as elements of a uniform buffer, or directly as push constants:
ident_t sh_var_int(pl_shader sh, const char *name, int val, bool dynamic);\nident_t sh_var_uint(pl_shader sh, const char *name, unsigned int val, bool dynamic);\nident_t sh_var_float(pl_shader sh, const char *name, float val, bool dynamic);\n#define SH_INT_DYN(val) sh_var_int(sh, \"const\", val, true)\n#define SH_UINT_DYN(val) sh_var_uint(sh, \"const\", val, true)\n#define SH_FLOAT_DYN(val) sh_var_float(sh, \"const\", val, true)\nThese are used primarily when a variable is expected to change very frequently, e.g. as a result of randomness, or for constants which depend on dynamically computed, source-dependent variables (e.g. input frame characteristics):
if (params->show_clipping) {\nconst float eps = 1e-6f;\nGLSL(\"bool clip_hi, clip_lo; \\n\"\n\"clip_hi = any(greaterThan(color.rgb, vec3(\"$\"))); \\n\"\n\"clip_lo = any(lessThan(color.rgb, vec3(\"$\"))); \\n\"\n\"clip_hi = clip_hi || ipt.x > \"$\"; \\n\"\n\"clip_lo = clip_lo || ipt.x < \"$\"; \\n\",\nSH_FLOAT_DYN(pl_hdr_rescale(PL_HDR_PQ, PL_HDR_NORM, tone.input_max) + eps),\nSH_FLOAT(pl_hdr_rescale(PL_HDR_PQ, PL_HDR_NORM, tone.input_min) - eps),\nSH_FLOAT_DYN(tone.input_max + eps),\nSH_FLOAT(tone.input_min - eps));\n}\nShader macros come in three main flavors, depending on where the resulting text should be formatted:
GLSL: Expanded in the scope of the current main function, and is related to code directly processing the current pixel value.GLSLH: Printed to the 'header', before the first function, but after variables, uniforms etc. This is used for global definitions, helper functions, shared memory variables, and so on.GLSLF: Printed to the footer, which is always at the end of the current main function, but before returning to the caller / writing to the framebuffer. Used to e.g. update SSBO state in preparation for the next frame.Finally, there is a fourth category GLSLP (prelude), which is currently only used internally to generate preambles during e.g. compute shader translation.
Starting with libplacebo v6, the internal shader system has been augmented by a custom macro preprocessor, which is designed to ease the boilerplate of writing shaders (and also strip redundant whitespace from generated shaders). The code for this is found in the tools/glsl_preproc directory.
In a nutshell, this allows us to embed GLSL snippets directly as #pragma GLSL macros (resp. #pragma GLSLH, #pragma GLSLF):
bool pl_shader_sample_bicubic(pl_shader sh, const struct pl_sample_src *src)\n{\nident_t tex, pos, pt;\nfloat rx, ry, scale;\nif (!setup_src(sh, src, &tex, &pos, &pt, &rx, &ry, NULL, &scale, true, LINEAR))\nreturn false;\nif (rx < 1 || ry < 1) {\nPL_TRACE(sh, \"Using fast bicubic sampling when downscaling. This \"\n\"will most likely result in nasty aliasing!\");\n}\n// Explanation of how bicubic scaling with only 4 texel fetches is done:\n// http://www.mate.tue.nl/mate/pdfs/10318.pdf\n// 'Efficient GPU-Based Texture Interpolation using Uniform B-Splines'\nsh_describe(sh, \"bicubic\");\n#pragma GLSL /* pl_shader_sample_bicubic */ \\\n vec4 color; \\\n { \\\n vec2 pos = $pos; \\\n vec2 size = vec2(textureSize($tex, 0)); \\\n vec2 frac = fract(pos * size + vec2(0.5)); \\\n vec2 frac2 = frac * frac; \\\n vec2 inv = vec2(1.0) - frac; \\\n vec2 inv2 = inv * inv; \\\n/* compute basis spline */ \\\n vec2 w0 = 1.0/6.0 * inv2 * inv; \\\n vec2 w1 = 2.0/3.0 - 0.5 * frac2 * (2.0 - frac); \\\n vec2 w2 = 2.0/3.0 - 0.5 * inv2 * (2.0 - inv); \\\n vec2 w3 = 1.0/6.0 * frac2 * frac; \\\n vec4 g = vec4(w0 + w1, w2 + w3); \\\n vec4 h = vec4(w1, w3) / g + inv.xyxy; \\\n h.xy -= vec2(2.0); \\\n/* sample four corners, then interpolate */ \\\n vec4 p = pos.xyxy + $pt.xyxy * h; \\\n vec4 c00 = textureLod($tex, p.xy, 0.0); \\\n vec4 c01 = textureLod($tex, p.xw, 0.0); \\\n vec4 c0 = mix(c01, c00, g.y); \\\n vec4 c10 = textureLod($tex, p.zy, 0.0); \\\n vec4 c11 = textureLod($tex, p.zw, 0.0); \\\n vec4 c1 = mix(c11, c10, g.y); \\\n color = ${float:scale} * mix(c1, c0, g.x); \\\n }\nreturn true;\n}\nThis gets transformed, by the GLSL macro preprocessor, into an optimized shader template invocation like the following:
{\n// ...\nsh_describe(sh, \"bicubic\");\nconst struct __attribute__((__packed__)) {\nident_t pos;\nident_t tex;\nident_t pt;\nident_t scale;\n} _glsl_330_args = {\n.pos = pos,\n.tex = tex,\n.pt = pt,\n.scale = sh_const_float(sh, \"scale\", scale),\n};\nsize_t _glsl_330_fn(void *, pl_str *, const uint8_t *);\npl_str_builder_append(sh->buffers[SH_BUF_BODY], _glsl_330_fn,\n&_glsl_330_args, sizeof(_glsl_330_args));\n// ...\n}\nsize_t _glsl_330_fn(void *alloc, pl_str *buf, const uint8_t *ptr)\n{\nstruct __attribute__((__packed__)) {\nident_t pos;\nident_t tex;\nident_t pt;\nident_t scale;\n} vars;\nmemcpy(&vars, ptr, sizeof(vars));\npl_str_append_asprintf(alloc, buf,\n\"/* pl_shader_sample_bicubic */\\n\"\n\" vec4 color;\\n\"\n\" {\\n\"\n\" vec2 pos = /*pos*/_%hx;\\n\"\n\" vec2 size = vec2(textureSize(/*tex*/_%hx, 0));\\n\"\n\" vec2 frac = fract(pos * size + vec2(0.5));\\n\"\n\" vec2 frac2 = frac * frac;\\n\"\n\" vec2 inv = vec2(1.0) - frac;\\n\"\n\" vec2 inv2 = inv * inv;\\n\"\n\" /* compute basis spline */\\n\"\n\" vec2 w0 = 1.0/6.0 * inv2 * inv;\\n\"\n\" vec2 w1 = 2.0/3.0 - 0.5 * frac2 * (2.0 - frac);\\n\"\n\" vec2 w2 = 2.0/3.0 - 0.5 * inv2 * (2.0 - inv);\\n\"\n\" vec2 w3 = 1.0/6.0 * frac2 * frac;\\n\"\n\" vec4 g = vec4(w0 + w1, w2 + w3);\\n\"\n\" vec4 h = vec4(w1, w3) / g + inv.xyxy;\\n\"\n\" h.xy -= vec2(2.0);\\n\"\n\" /* sample four corners, then interpolate */\\n\"\n\" vec4 p = pos.xyxy + /*pt*/_%hx.xyxy * h;\\n\"\n\" vec4 c00 = textureLod(/*tex*/_%hx, p.xy, 0.0);\\n\"\n\" vec4 c01 = textureLod(/*tex*/_%hx, p.xw, 0.0);\\n\"\n\" vec4 c0 = mix(c01, c00, g.y);\\n\"\n\" vec4 c10 = textureLod(/*tex*/_%hx, p.zy, 0.0);\\n\"\n\" vec4 c11 = textureLod(/*tex*/_%hx, p.zw, 0.0);\\n\"\n\" vec4 c1 = mix(c11, c10, g.y);\\n\"\n\" color = /*scale*/_%hx * mix(c1, c0, g.x);\\n\"\n\" }\\n\",\nvars.pos,\nvars.tex,\nvars.pt,\nvars.tex,\nvars.tex,\nvars.tex,\nvars.tex,\nvars.scale\n);\nreturn sizeof(vars);\n}\nTo support this style of shader programming, special syntax was invented:
"},{"location":"glsl/#shader-variables","title":"Shader variables","text":"Instead of being formatted with \"$\", %f etc. and supplied in a big list, printf style, GLSL macros may directly embed shader variables:
ident_t pos, tex = sh_bind(sh, texture, ..., &pos, ...);\n#pragma GLSL vec4 color = texture($tex, $pos);\nThe simplest possible shader variable is just $name, which corresponds to any variable of type ident_t. More complicated expression are also possible:
#define RAND3 ${sh_prng(sh, false, NULL)}\ncolor.rgb += ${float:params->noise} * RAND3;\nIn the expression ${float:params->noise}, the float: prefix here transforms the shader variable into the equivalent of SH_FLOAT() in the legacy API, that is, a generic float (specialization) constant. Other possible types are:
float f = ${float: M_PI};\nint i = ${int: params->width};\nuint u = ${uint: sizeof(ssbo)};\nIn addition to a type specifier, the optional qualifiers dynamic and const will modify the variable, turning it into (respectively) a dynamically loaded uniform (SH_FLOAT_DYN etc.), or a hard-coded shader literal (%d, %f etc.):
const float base = ${const float: M_LOG10E};\nint seed = ${dynamic int: rand()};\nLines beginning with @ are not included in the GLSL as-is, but instead parsed as macro directives, to control the code flow inside the macro expansion:
Standard-purpose conditional. Example:
float alpha = ...;\n@if (repr.alpha == PL_ALPHA_INDEPENDENT)\ncolor.a *= alpha;\n@else\ncolor.rgba *= alpha;\nThe condition is evaluated outside the macro (in the enclosing scope) and the resulting boolean variable is directly passed to the template.
An @if block can also enclose multiple lines:
@if (threshold > 0) {\nfloat thresh = ${float:threshold};\ncoeff = mix(coeff, vec2(0.0), lessThan(coeff, vec2(thresh)));\ncoeff = mix(coeff, vec2(1.0), greaterThan(coeff, vec2(1.0 - thresh)));\n@}\nThis can be used to generate (unrolled) loops:
int offset = ${const int: params->kernel_width / 2};\nfloat sum = 0.0;\n@for (x < params->kernel_width)\nsum += textureLodOffset($luma, $pos, 0.0, int(@sum - offset)).r;\nThis introduces a local variable, @x, which expands to an integer containing the current loop index. Loop indices always start at 0. Valid terminating conditions include < and <=, and the loop stop condition is also evaluated as an integer.
Alternatively, this can be used to iterate over a bitmask (as commonly used for e.g. components in a color mask):
float weight = /* ... */;\nvec4 color = textureLod($tex, $pos, 0.0);\n@for (c : params->component_mask)\nsum[@c] += weight * color[@c];\nFinally, to combine loops with conditionals, the special syntax @if @(cond) may be used to evaluate expressions inside the template loop:
@for (i < 10) {\nfloat weight = /* ... */;\n@if @(i < 5)\nweight = -weight;\nsum += weight * texture(...);\n@}\nIn this case, the @if conditional may only reference local (loop) variables.
This corresponds fairly straightforwardly to a normal switch/case from C:
@switch (color->transfer) {\n@case PL_COLOR_TRC_SRGB:\ncolor.rgb = mix(color.rgb * 1.0/12.92,\npow((color.rgb + vec3(0.055)) / 1.055, vec3(2.4)),\nlessThan(vec3(0.04045), color.rgb));\n@break;\n@case PL_COLOR_TRC_GAMMA18:\ncolor.rgb = pow(color.rgb, vec3(1.8));\n@break;\n@case PL_COLOR_TRC_GAMMA20:\ncolor.rgb = pow(color.rgb, vec3(2.0));\n@break;\n@case PL_COLOR_TRC_GAMMA22:\ncolor.rgb = pow(color.rgb, vec3(2.2));\n@break;\n/* ... */\n@}\nThe switch body is always evaluated as an unsigned int.
This example roughly builds off the previous entry, and as such will not cover the basics of how to create a window, initialize a pl_gpu and get pixels onto the screen.
The pl_renderer set of APIs represents the highest-level interface into libplacebo, and is what most users who simply want to display e.g. a video feed on-screen will want to be using.
The basic initialization is straightforward, requiring no extra parameters:
pl_renderer renderer;\ninit()\n{\nrenderer = pl_renderer_create(pllog, gpu);\nif (!renderer)\ngoto error;\n// ...\n}\nuninit()\n{\npl_renderer_destroy(&renderer);\n}\nWhat makes the renderer powerful is the large number of pl_render_params it exposes. By default, libplacebo provides several presets to use:
Covering all of the possible options exposed by pl_render_params is out-of-scope of this example and would be better served by looking at the API documentation.
pl_frame is the struct libplacebo uses to group textures and their metadata together into a coherent unit that can be rendered using the renderer. This is not currently a dynamically allocated or refcounted heap object, it is merely a struct that can live on the stack (or anywhere else). The actual data lives in corresponding pl_tex objects referenced in each of the frame's planes.
bool render_frame(const struct pl_frame *image,\nconst struct pl_swapchain_frame *swframe)\n{\nstruct pl_frame target;\npl_frame_from_swapchain(&target, swframe);\nreturn pl_render_image(renderer, image, target,\n&pl_render_default_params);\n}\nRenderer state
The pl_renderer is conceptually (almost) stateless. The only thing that is needed to get a different result is to change the render params, which can be varied freely on every call, if the user desires.
The one case where this is not entirely true is when using frame mixing (see below), or when using HDR peak detection. In this case, the renderer can be explicitly reset using pl_renderer_flush_cache.
To upload frames, the easiest methods are made available as dedicated helpers in <libplacebo/utils/upload.h>, and <libplacebo/utils/libav.h> (for AVFrames). In general, I recommend checking out the demo programs for a clearer illustration of how to use them in practice.
The renderer internally generates, compiles and caches a potentially large number of shader programs, some of which can be complex. On some platforms (notably D3D11), these can be quite costly to recompile on every program launch.
As such, the renderer offers a way to save/restore its internal shader cache from some external location (managed by the API user). The use of this API is highly recommended:
static uint8_t *load_saved_cache();\nstatic void store_saved_cache(uint8_t *cache, size_t bytes);\nvoid init()\n{\nrenderer = pl_renderer_create(pllog, gpu);\nif (!renderer)\ngoto error;\nuint8_t *cache = load_saved_cache();\nif (cache) {\npl_renderer_load(renderer, cache);\nfree(cache);\n}\n// ...\n}\nvoid uninit()\n{\nsize_t cache_bytes = pl_renderer_save(renderer, NULL);\nuint8_t *cache = malloc(cache_bytes);\nif (cache) {\npl_renderer_save(renderer, cache);\nstore_saved_cache(cache, cache_bytes);\nfree(cache);\n}\npl_renderer_destroy(&renderer);\n}\nCache safety
libplacebo performs only minimal validity checking on the shader cache, and in general, cannot possibly guard against malicious alteration of such files. Loading a cache from an untrusted source represents a remote code execution vector.
"},{"location":"renderer/#frame-mixing","title":"Frame mixing","text":"One of the renderer's most powerful features is its ability to compensate for differences in framerates between the source and display by using frame mixing to blend adjacent frames together.
Using this API requires presenting the renderer, at each vsync, with a pl_frame_mix struct, describing the current state of the vsync. In principle, such structs can be constructed by hand. To do this, all of the relevant frames (nearby the vsync timestamp) must be collected, and their relative distances to the vsync determined, by normalizing all PTS values such that the vsync represents time 0.0 (and a distance of 1.0 represents the nominal duration between adjacent frames). Note that timing vsyncs, and determining the correct vsync duration, are both left as problems for the user to solve.1. Here could be an example of a valid struct:
(struct pl_frame_mix) {\n.num_frames = 6\n.frames = (const struct pl_frame *[]) {\n/* frame 0 */\n/* frame 1 */\n/* ... */\n/* frame 5 */\n},\n.signatures = (uint64_t[]) {\n0x0, 0x1, 0x2, 0x3, 0x4, 0x5 // (1)\n},\n.timestamps = (float[]) {\n-2.4, -1.4, -0.4, 0.6, 1.6, 2.6, // (2)\n},\n.vsync_duration = 0.4, // 24 fps video on 60 fps display\n}\nThese must be unique per frame, but always refer to the same frame. For example, this could be based on the frame's PTS, the frame's numerical ID (in order of decoding), or some sort of hash. The details don't matter, only that this uniquely identifies specific frames.
Typically, for CFR sources, frame timestamps will always be separated in this list by a distance of 1.0. In this example, the vsync falls roughly halfway (but not quite) in between two adjacent frames (with IDs 0x2 and 0x3).
Frame mixing radius
In this example, the frame mixing radius (as determined by pl_frame_mix_radius is 3.0, so we include all frames that fall within the timestamp interval of [-3, 3). In general, you should consult this function to determine what frames need to be included in the pl_frame_mix - though including more frames than needed is not an error.
Because this API is rather unwieldy and clumsy to use directly, libplacebo provides a helper abstraction known as pl_queue to assist in transforming some arbitrary source of frames (such as a video decoder) into nicely packed pl_frame_mix structs ready for consumption by the pl_renderer:
#include <libplacebo/utils/frame_queue.h>\npl_queue queue;\nvoid init()\n{\nqueue = pl_queue_create(gpu);\n}\nvoid uninit()\n{\npl_queue_destroy(&queue);\n// ...\n}\nThis queue can be interacted with through a number of mechanisms: either pushing frames (blocking or non-blocking), or by having the queue poll frames (via blocking or non-blocking callback) as-needed. For a full overview of the various methods of pushing and polling frames, check the API documentation.
In this example, I will assume that we have a separate decoder thread pushing frames into the pl_queue in a blocking manner:
static void decoder_thread(void)\n{\nvoid *frame;\nwhile ((frame = /* decode new frame */)) {\npl_queue_push_block(queue, UINT64_MAX, &(struct pl_source_frame) {\n.pts = /* frame pts */,\n.duration = /* frame duration */,\n.map = /* map callback */,\n.unmap = /* unmap callback */,\n.frame_data = frame,\n});\n}\npl_queue_push(queue, NULL); // signal EOF\n}\nNow, in our render loop, we want to call pl_queue_update with appropriate values to retrieve the correct frame mix for each vsync:
bool render_frame(const struct pl_swapchain_frame *swframe)\n{\nstruct pl_frame_mix mix;\nenum pl_queue_status res;\nres = pl_queue_update(queue, &mix, pl_queue_params(\n.pts = /* time of next vsync */,\n.radius = pl_frame_mix_radius(&render_params),\n.vsync_duration = /* if known */,\n.timeout = UINT64_MAX, // (2)\n));\nswitch (res) {\ncase PL_QUEUE_OK:\nbreak;\ncase PL_QUEUE_EOF:\n/* no more frames */\nreturn false;\ncase PL_QUEUE_ERR:\ngoto error;\n// (1)\n}\nstruct pl_frame target;\npl_frame_from_swapchain(&target, swframe);\nreturn pl_render_image_mix(renderer, &mix, target,\n&pl_render_default_params);\n}\nThere is a fourth status, PL_QUEUE_MORE, which is returned only if the resulting frame mix is incomplete (and the timeout was reached) - basically this can only happen if the queue runs dry due to frames not being supplied fast enough.
In this example, since we are setting timeout to UINT64_MAX, we will never get this return value.
Setting this makes pl_queue_update block indefinitely until sufficiently many frames have been pushed into the pl_queue from our separate decoding thread.
The frame queue also vastly simplifies the process of performing motion-adaptive temporal deinterlacing, by automatically linking together adjacent fields/frames. To take advantage of this, all you need to do is set the appropriate field (pl_source_frame.first_frame), as well as enabling deinterlacing parameters.
However, this may change in the future, as the recent introduction of the Vulkan display timing extension may result in display timing feedback being added to the pl_swapchain API. That said, as of writing, this has not yet happened.\u00a0\u21a9