[BUG] Silently crashes when encoding at presets 6 or lower

[muizzsiddique]

Overview
Encoding at or below --preset 6 will encode my test video for up to 83 frames (Encoding shows 1, 23-27 or 84 typically) before crashing. It doesn't produce any error information.

Branch
In which branch does the issue appear to be occurring?

• master
• testing

Reproduction
Steps to reproduce the behavior:

1. Build Debug build of application
2. Run svtav1encapp --preset 6 --input in.y4m --output out.ivf
3. Watch it encode up to 84 frames
4. See no output when it crashes

Expected behavior
Encode the video.

Terminal Output

My terminal output

PS D:\Videos\Captures> svtav1encapp-git-debug --preset 6 --input in2.y4m --output out.ivf
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:       SVT-AV1-PSY Encoder Lib v2.3.0-A
Svt[info]: SVT [build]  :       GCC 14.2.0       64 bit
Svt[info]: LIB Build date: Dec 26 2024 13:28:21
Svt[info]: -------------------------------------------
Svt[info]: Level of Parallelism: 5
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto)     level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator               : 1440 / 1552 / 60 / 1
Svt[info]: SVT [config]: bit-depth / color format                                       : 8 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                                    : 6 / SSIM / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type                      : 641 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor                                         : CRF / 35.00
Svt[info]: SVT [config]: AQ mode / variance boost strength / octile / curve             : 2 / 2 / 6 / regular
Svt[info]: SVT [config]: Sharpness / QP scale compress strength / Frame low-luma bias   : 1 / 1 / 0
Svt[info]: SVT [config]: Temporal Filtering Strength                                    : 1
Svt[info]: SVT [config]: Keyframe TF Strength                                           : 1
Svt[info]: -------------------------------------------
SvtMalloc[info]: SVT Memory Usage:
SvtMalloc[info]:     total allocated memory:       2.19 GB
SvtMalloc[info]:         malloced memory:          805.23 MB
SvtMalloc[info]:         callocated memory:        41.66 MB
SvtMalloc[info]:         allocated aligned memory: 1.36 GB
SvtMalloc[info]:     mutex count: 25329
SvtMalloc[info]:     semaphore count: 681
SvtMalloc[info]:     thread count: 74
SvtMalloc[info]:     hash table fulless: 0.142434, hash bucket is healthy
SvtMalloc[info]: top 10 malloced memory locations:
SvtMalloc[info]: (305.47 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/coding_unit.c:84
SvtMalloc[info]: (98.83 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:1075
SvtMalloc[info]: (79.13 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:1528
SvtMalloc[info]: (45.81 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/bitstream_unit.c:36
SvtMalloc[info]: (45.78 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/bitstream_unit.c:36
SvtMalloc[info]: (43.96 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:1531
SvtMalloc[info]: (30.40 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:114
SvtMalloc[info]: (19.76 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:115
SvtMalloc[info]: (17.97 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:1047
SvtMalloc[info]: (13.67 MB): C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/pcs.c:1312
Encoding        84

Platform:

• Operating System: Windows 10 21H2 OS build 19044.5247
• Clang/GCC Version: clang version 19.1.6
• Build Command: cmake .. -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=OFF -DBUILD_DEC=OFF -DSVT_AV1_LTO=ON -DNATIVE=ON -DCMAKE_CXX_FLAGS="-O3" -DCMAKE_C_FLAGS="-O3" -DCMAKE_LD_FLAGS="-O3" && ninja

Version String (please complete the following information):

• Run SvtAv1EncApp --version and copy the result: SVT-AV1-PSY v2.3.0-A (debug)

Additional context / Relevant Files

This does not happen on my build of v2.3.0, only v2.3.0-A and the master branch at the time.

My test video that fails within 84 frames: https://drive.google.com/file/d/1qG1x4uuiah0FRD5qkrfoA3tqr9DnsiTD/view?usp=sharing

My build log

Muizz@MuizzXMG-LTSC UCRT64 ~/svt-av1-psy-git/svt_build
$ cmake .. -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=OFF -DBUILD_DEC=OFF -DSVT_AV1_LTO=ON -DNATIVE=ON -DCMAKE_CXX_FLAGS="-O3" -DCMAKE_C_FLAGS="-O3" -DCMAKE_LD_FLAGS="-O3" && ninja
-- Building for: Ninja
-- The C compiler identification is GNU 14.2.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/msys64/ucrt64/bin/cc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/msys64/ucrt64/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_X86_PLATFORM
-- Performing Test HAVE_X86_PLATFORM - Success
-- Performing Test HAVE_ARM_PLATFORM
-- Performing Test HAVE_ARM_PLATFORM - Failed
-- Looking for a ASM_NASM compiler
-- Looking for a ASM_NASM compiler - C:/msys64/ucrt64/bin/yasm.exe
-- The ASM_NASM compiler identification is YASM
-- Found assembler: C:/msys64/ucrt64/bin/yasm.exe
-- BUILD_SHARED_LIBS: OFF
-- Checking C flag support for: [-Wformat-security] - Yes
-- Checking C flag support for: [-Wformat] - Yes
-- Checking C flag support for: [-Wextra] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CFLAGS: -Wextra -Wformat -Wformat-security -O3
-- Checking CXX flag support for: [-Wformat-security] - Yes
-- Checking CXX flag support for: [-Wformat] - Yes
-- Checking CXX flag support for: [-Wextra] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CXXFLAGS: -Wextra -Wformat -Wformat-security -O3
-- Checking C flag support for: [-Wall] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3
-- Checking CXX flag support for: [-Wall] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3
-- Checking C flag support for: [-march=native] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native
-- Checking CXX flag support for: [-march=native] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native
-- Checking C flag support for: [-mxsave] - Yes
-- Checking C flag support for: [-fno-asynchronous-unwind-tables] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables
-- Checking C flag support for: [-mno-avx] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx
-- Checking CXX flag support for: [-mno-avx] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx
-- Performing Test HAS_AVX512
-- Performing Test HAS_AVX512 - Success
-- Looking for strnlen_s
-- Looking for strnlen_s - not found
-- Looking for strncpy_s
-- Looking for strncpy_s - found
-- Looking for strcpy_s
-- Looking for strcpy_s - found
-- Looking for valgrind/valgrind.h
-- Looking for valgrind/valgrind.h - not found
-- Performing Test HAVE_BUILTIN_EXPECT
-- Performing Test HAVE_BUILTIN_EXPECT - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for elf_aux_info
-- Looking for elf_aux_info - not found
-- Checking C flag support for: [-msse2] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2 CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx -msse2
-- Checking CXX flag support for: [-msse2] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2 CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx -msse2
-- Checking C flag support for: [-mssse3] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSSE3 CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx -mssse3
-- Checking CXX flag support for: [-mssse3] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSSE3 CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx -mssse3
-- Checking C flag support for: [-msse4.1] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE4_1 CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx -msse4.1
-- Checking CXX flag support for: [-msse4.1] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE4_1 CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx -msse4.1
-- Checking C flag support for: [-mavx2] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX2 CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx -mavx2
-- Checking CXX flag support for: [-mavx2] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX2 CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx -mavx2
-- Checking C flag support for: [-mavx512f] - Yes
-- Checking C flag support for: [-mavx512bw] - Yes
-- Checking C flag support for: [-mavx512dq] - Yes
-- Checking C flag support for: [-mavx512vl] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX512 CFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mxsave -fno-asynchronous-unwind-tables -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq -mavx512vl
-- Checking CXX flag support for: [-mavx512f] - Yes
-- Checking CXX flag support for: [-mavx512bw] - Yes
-- Checking CXX flag support for: [-mavx512dq] - Yes
-- Checking CXX flag support for: [-mavx512vl] - Yes
-- C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX512 CXXFLAGS: -Wall -Wextra -Wformat -Wformat-security -O3 -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq -mavx512vl
-- Found PkgConfig: C:/msys64/ucrt64/bin/pkg-config.exe (found version "2.3.0")
-- Building with dovi support - No
-- Building with hdr10plus support - No
-- Configuring done (17.1s)
-- Generating done (0.2s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_DEC
    CMAKE_LD_FLAGS


-- Build files have been written to: C:/msys64/home/Muizz/svt-av1-psy-git/svt_build
[6/269] Generating version header
-- Configured version: v2.3.0-A
[114/269] Building C object Source/Lib/Codec/CMakeFiles/CODEC.dir/svt_threads.c.obj
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/svt_threads.c: In function 'svt_create_thread':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/svt_threads.c:128:9: warning: cast between incompatible function types from 'void * (*)(void *)' to 'DWORD (*)(void *)' {aka 'long unsigned int (*)(void *)'} [-Wcast-function-type]
  128 |         (LPTHREAD_START_ROUTINE)thread_function, // function to be tied to the new thread
      |         ^
[259/269] Building C object Source/App/CMakeFiles/SvtAv1EncApp.dir/app_config.c.obj
C:/msys64/home/Muizz/svt-av1-psy-git/Source/App/app_config.c: In function 'set_cfg_force_key_frames':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/App/app_config.c:498:55: warning: 'calloc' sizes specified with 'sizeof' in the earlier argument and not in the later argument [-Wcalloc-transposed-args]
  498 |         char        *specifier = (char *)calloc(sizeof(*specifier), len + 1);
      |                                                       ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/App/app_config.c:498:55: note: earlier argument should specify number of elements, later size of each element
C:/msys64/home/Muizz/svt-av1-psy-git/Source/App/app_config.c:517:43: warning: 'calloc' sizes specified with 'sizeof' in the earlier argument and not in the later argument [-Wcalloc-transposed-args]
  517 |     fkf.frames = (uint64_t *)calloc(sizeof(*fkf.frames), fkf.count);
      |                                           ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/App/app_config.c:517:43: note: earlier argument should specify number of elements, later size of each element
[263/269] Building C object third_party/cpuinfo/CMakeFiles/cpuinfo.dir/src/x86/windows/init.c.obj
C:/msys64/home/Muizz/svt-av1-psy-git/third_party/cpuinfo/src/x86/windows/init.c: In function 'cpuinfo_x86_windows_init':
C:/msys64/home/Muizz/svt-av1-psy-git/third_party/cpuinfo/src/x86/windows/init.c:21:51: warning: unused parameter 'init_once' [-Wunused-parameter]
   21 | BOOL CALLBACK cpuinfo_x86_windows_init(PINIT_ONCE init_once, PVOID parameter, PVOID* context) {
      |                                        ~~~~~~~~~~~^~~~~~~~~
C:/msys64/home/Muizz/svt-av1-psy-git/third_party/cpuinfo/src/x86/windows/init.c:21:68: warning: unused parameter 'parameter' [-Wunused-parameter]
   21 | BOOL CALLBACK cpuinfo_x86_windows_init(PINIT_ONCE init_once, PVOID parameter, PVOID* context) {
      |                                                              ~~~~~~^~~~~~~~~
C:/msys64/home/Muizz/svt-av1-psy-git/third_party/cpuinfo/src/x86/windows/init.c:21:86: warning: unused parameter 'context' [-Wunused-parameter]
   21 | BOOL CALLBACK cpuinfo_x86_windows_init(PINIT_ONCE init_once, PVOID parameter, PVOID* context) {
      |                                                                               ~~~~~~~^~~~~~~
[269/269] Linking C executable C:\msys64\home\Muizz\svt-av1-psy-git\Bin\Debug\SvtAv1EncApp.exe
lto1.exe: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
In function '_mm_add_epi64',
    inlined from 'compute_stats_win5_avx512' at C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX512/pickrst_avx512.c:2162:36:
C:/msys64/ucrt64/lib/gcc/x86_64-w64-mingw32/14.2.0/include/emmintrin.h:1078:21: warning: 'delta4' may be used uninitialized [-Wmaybe-uninitialized]
 1078 |   return (__m128i) ((__v2du)__A + (__v2du)__B);
      |                     ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX512/pickrst_avx512.c: In function 'compute_stats_win5_avx512':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX512/pickrst_avx512.c:1980:38: note: 'delta4' was declared here
 1980 |             __m128i        delta128, delta4;
      |                                      ^
In function '_mm_storeu_si128',
    inlined from 'svt_enc_msb_un_pack2d_sse2_intrin' at C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c:116:17:
C:/msys64/ucrt64/lib/gcc/x86_64-w64-mingw32/14.2.0/include/emmintrin.h:742:8: warning: 'temp_pixel0_u8' may be used uninitialized [-Wmaybe-uninitialized]
  742 |   *__P = __B;
      |        ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c: In function 'svt_enc_msb_un_pack2d_sse2_intrin':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c:27:13: note: 'temp_pixel0_u8' was declared here
   27 |     __m128i temp_pixel0_u8, temp_pixel1_u8;
      |             ^
In function '_mm_storeu_si128',
    inlined from 'svt_enc_msb_un_pack2d_sse2_intrin' at C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c:117:17:
C:/msys64/ucrt64/lib/gcc/x86_64-w64-mingw32/14.2.0/include/emmintrin.h:742:8: warning: 'temp_pixel1_u8' may be used uninitialized [-Wmaybe-uninitialized]
  742 |   *__P = __B;
      |        ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c: In function 'svt_enc_msb_un_pack2d_sse2_intrin':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_SSE2/pack_unpack_intrin_sse2.c:27:29: note: 'temp_pixel1_u8' was declared here
   27 |     __m128i temp_pixel0_u8, temp_pixel1_u8;
      |                             ^
In function '_mm_add_epi64',
    inlined from 'compute_stats_win5_avx2' at C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX2/pickrst_avx2.c:2024:36:
C:/msys64/ucrt64/lib/gcc/x86_64-w64-mingw32/14.2.0/include/emmintrin.h:1078:21: warning: 'delta4' may be used uninitialized [-Wmaybe-uninitialized]
 1078 |   return (__m128i) ((__v2du)__A + (__v2du)__B);
      |                     ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX2/pickrst_avx2.c: In function 'compute_stats_win5_avx2':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/ASM_AVX2/pickrst_avx2.c:1840:38: note: 'delta4' was declared here
 1840 |             __m128i        delta128, delta4;
      |                                      ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/rc_process.c: In function 'svt_aom_rate_control_kernel':
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/rc_process.c:3300:13: warning: 'scs' may be used uninitialized [-Wmaybe-uninitialized]
 3300 |             assert(scs->static_config.superres_mode == SUPERRES_QTHRESH ||
      |             ^
C:/msys64/home/Muizz/svt-av1-psy-git/Source/Lib/Codec/rc_process.c:3277:25: note: 'scs' was declared here
 3277 |     SequenceControlSet *scs;
      |                         ^

[Dendraspis]

Same for me, but it also crashed with --preset 7.

The workaround is to set the bit-depth of the source file to 10-bit instead of 8-bit, which let's the encode run smoothly to the end.
Nice side effect: Reduction of color-banding and such.

[gitoss]

The workaround is to set the bit-depth of the source file to 10-bit instead of 8-bit, which let's the encode run smoothly to the end. Nice side effect: Reduction of color-banding and such.

Alas, 8-bit is slightly faster, so though 10-bit decoding capability is mandatory for av1 there'S still some justification to use it.

[Dendraspis]

And that's the difference between "workaround" and "solution". 😅

I'm sure the guys will come up with a fix soon...

[gitoss]

And that's the difference between "workaround" and "solution". 😅

Sure, I've understood that :-> ... though that the 8-bit crashes were under the radar for so long shows that most people around here encode at 10-bit.

[muizzsiddique]

How long have the 8-bit crashes been under the radar for? I swear this has only been an issue since the A micro update.

[gitoss]

How long have the 8-bit crashes been under the radar for? I swear this has only been an issue since the A micro update.

Personally, I wouldn't know because I only encode in 10-bit.

However, if you didn't experience any crashes before the micro update, the other bug tickets indicate that this might not be 100% definitve.

The current problems seem to be a dev's worst nightmare: A problable general instability, crashes occur only sometimes and under not-yet-determined conditions, analytics show issues for a long time.

So with the 8-bit you could be lucky: The app seems to crash rather quickly and reliabably :-)