[BUG] regression from 916cabd50bba to v2.3.0-A: crash #114

vi · 2024-12-25T16:57:12Z

Overview

ffmpeg equipped with svt-av1-spy crashes when using low tune, high crf in v2.3.0-A, though it worked before.

Branch
In which branch does the issue appear to be occurring?

master
testing

Reproduction

Build FFmpeg with SVG-AV1-PSY's version of prefix (I actually ffmpeg executable from older version of the library, hoping that ABI is the same)
Try encoding a 1080p video with -g 1800 -c:v libsvtav1 -svtav1-params preset=2:crf=70 parameters.

Expected behavior

It produced the video

Terminal Output
If applicable, add terminal output to help explain your problem.

Part of FFmpeg output

Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:       SVT-AV1-PSY Encoder Lib v2.3.0-A
Svt[info]: SVT [build]  :       GCC 12.2.0       64 bit
Svt[info]: LIB Build date: Dec 25 2024 17:35:00
Svt[info]: -------------------------------------------
Svt[info]: Level of Parallelism: 5
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto)     level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator               : 1920 / 1080 / 30 / 1
Svt[info]: SVT [config]: bit-depth / color format                                       : 8 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                                    : 2 / SSIM / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type                      : 1800 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor                                         : CRF / 70.00
Svt[info]: SVT [config]: AQ mode / variance boost strength / octile / curve             : 2 / 2 / 6 / regular
Svt[info]: SVT [config]: Sharpness / QP scale compress strength / Frame low-luma bias   : 1 / 1 / 0
Svt[info]: SVT [config]: Temporal Filtering Strength                                    : 1
Svt[info]: SVT [config]: Keyframe TF Strength                                           : 1
Svt[info]: -------------------------------------------
Output #0, webm, to 'q.webm':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: av1, yuv420p(tv, progressive), 1920x1080, q=2-31, 30 fps, 1k tbn
      Metadata:
        DURATION        : 00:00:07.633000000
        encoder         : Lavc61.19.100 libsvtav1
Segmentation fault1 q=47.0 size=       0KiB time=00:00:00.00 bitrate=N/A speed=   0x
(it successfully encodes frame 0 and shows frame 1 for a moment)

Platform (please complete the following information):

Operating System Linux hostname 6.1.0-23-amd64 x86_64 unknown GNU/Linux
Clang/GCC Version gcc (Debian 12.2.0-14) 12.2.0, Debian clang version 14.0.6 (haven't check which is used for the build)
Build Command (./build.sh [args] that you used to build the problematic binary) - not sure - I am reusing CMake/Ninja build directory after git pull.

Version String (please complete the following information):

Run SvtAv1EncApp --version and copy the result: SVT-AV1-PSY v2.3.0-A (release) PSY Release: A

Additional context / Relevant Files

git bisect points to this commit: 129faba

Built debug version (with the following patch to make it linkeable):

patch to make it build in debug mode

diff --git a/Source/Lib/Codec/psy_rd.c b/Source/Lib/Codec/psy_rd.c
index 46148616..9b661d9c 100644
--- a/Source/Lib/Codec/psy_rd.c
+++ b/Source/Lib/Codec/psy_rd.c
@@ -25,7 +25,7 @@
         d3 = t1 - t3; \
 }

-inline uint32_t ashft(uint32_t a);
+uint32_t ashft(uint32_t a);

 // 8-bit
 static int svt_sa8d_8x8(const uint8_t* s, uint32_t sp, const uint8_t* r, uint32_t rp);
@@ -40,7 +40,7 @@ static int svt_psy_sad_nxn_hbd(const uint8_t bw, const uint8_t bh, const uint16_
     uint32_t sp, const uint16_t* r, uint32_t rp);

 /* Performs absolute value operation quickly */
-inline uint32_t ashft(uint32_t a) {
+uint32_t ashft(uint32_t a) {
     uint32_t s = ((a >> (BITS_PER_SUM - 1)) & (((uint32_t)1 << BITS_PER_SUM) + 1)) * ((uint16_t)-1);
     return (a + s) ^ s;
 }

and obtained this backtrace:

Backtrace

#0  0x00007fffeb8ce6dd in _mm256_loadu_si256 (__P=0x7fff4bfdcff0) at /usr/lib/gcc/x86_64-linux-gnu/12/include/avxintrin.h:929
No locals.
#1  spatial_full_distortion_kernel32_avx2_intrin (input=0x7fff28a419e4 'k' <repeats 71 times>, "d", '\200' <repeats 128 times>..., recon=0x7fff4bfdcff0 "\020", sum=0x7fff014d3340) at /home/vi/src/svt-av1-psy/Source/Lib/ASM_AVX2/pic_operators_inline_avx2.h:170
        in = {7740398493674204011, 7740398493674204011, 7740398493674204011, 7740398493674204011}
        re = {671616, 7, 0, 17}
        max = {7740398493674204032, 7740398493674204011, 7740398493674204011, 7740398493674204011}
        min = {671595, 7, 0, 17}
        diff = {7740398493673532437, 7740398493674204004, 7740398493674204011, 7740398493674203994}
        diff_l = {30118239122751509, 30118282076553323, 30118282076553323, 30118282076553323}
        diff_h = {30118282076553316, 30118282076553323, 30118282076553306, 30118282076553323}
        dist_l = {89584427862345, 98346161166706, 98346161166706, 98346161166706}
        dist_h = {98346161165257, 98346161166706, 98346161163357, 98346161166706}
        dist = {187930589027602, 196692322333412, 196692322330063, 196692322333412}
#2  0x00007fffeb8dbc32 in svt_spatial_full_distortion_kernel_avx2 (input=0x7fff289ad184 "\232\224\224\224\224\224\224\224\224\224\224\223", '\224' <repeats 188 times>..., input_offset=0, input_stride=1032, recon=0x7fff4bf45070 '\210' <repeats 200 times>..., recon_offset=0, recon_stride=1056, area_width=1920, area_height=1080) at /home/vi/src/svt-av1-psy/Source/Lib/ASM_AVX2/pic_operators_intrin_avx2.c:1783
        w = 1536
        sum64 = {557332119, 556532332, 553064168, 553027989}
        inp = 0x7fff28a413e4 '\200' <repeats 60 times>, 'e' <repeats 37 times>, 'k' <repeats 103 times>...
        rec = 0x7fff4bfdc9f0 ""
        leftover = 0
        h = 492
        sum = {10380304596553799, 10517000520791296, 10161896919607669, 10747593020026398}
        sum_l = {-1446803456761533461, -1446803456761533461}
        sum_h = {-1446803456761533461, -1446803456761533461}
        s = {19060271, 20224532}
#3  0x00007fffeb04bad1 in picture_sse_calculations (pcs=0x7fff660714b0, recon_ptr=0x7fff4bf389a0, plane=1) at /home/vi/src/svt-av1-psy/Source/Lib/Codec/deblocking_filter.c:770
        sse = 9259542123273814144
        psy = 73887733298266240
        spatial_distortion = 73014444062
        input_pic = 0x7fff289a3fa0
        scs = 0x7fffd41c56d0
        is_16bit = 0 '\000'
        input_align_width = 1920
        input_align_height = 1080
        ss_x = 1
        ss_y = 1
        input_buffer = 0x7fff289ad184 "\232\224\224\224\224\224\224\224\224\224\224\223", '\224' <repeats 188 times>...
        recon_coeff_buffer = 0x7fff4bf45070 '\210' <repeats 200 times>...
#4  0x00007fffeb04c704 in try_filter_frame (sd=0x7fff289a3fa0, temp_lf_recon_buffer=0x7ffe64290ac0, pcs=0x7fff660714b0, filt_level=0, partial_frame=0, plane=1, dir=0) at /home/vi/src/svt-av1-psy/Source/Lib/Codec/deblocking_filter.c:935
        filt_err = 304001221190447171
        frm_hdr = 0x7fffd496cb10
        __PRETTY_FUNCTION__ = "try_filter_frame"
        filter_level = {0, 0}
        is_16bit = 0 '\000'
        recon_buffer = 0x7fff4bf389a0
#5  0x00007fffeb04c97c in search_filter_level (sd=0x7fff289a3fa0, temp_lf_recon_buffer=0x7ffe64290ac0, pcs=0x7fff660714b0, partial_frame=0, last_frame_filter_level=0x7fff014d3af0, best_cost_ret=0x0, plane=1, dir=0) at /home/vi/src/svt-av1-psy/Source/Lib/Codec/deblocking_filter.c:989
        min_filter_level = 0
        max_filter_level = 63
        filt_direction = 0
        best_err = 39284912
        filt_best = 6
        frm_hdr = 0x7fffd496cb10
        lvl = 0
        __PRETTY_FUNCTION__ = "search_filter_level"
        filt_mid = 0
        filter_step = 4
        is_16bit = 0 '\000'
        recon_buffer = 0x7fff4bf389a0
        ss_err = {-1 <repeats 64 times>}
        tot_convergence = 3
#6  0x00007fffeb04db46 in svt_av1_pick_filter_level (srcBuffer=0x7fff289a3fa0, pcs=0x7fff660714b0, method=LPF_PICK_FROM_FULL_IMAGE) at /home/vi/src/svt-av1-psy/Source/Lib/Codec/deblocking_filter.c:1299
        padding = 96
        temp_lf_recon_buffer = 0x7ffe64290ac0
        temp_lf_recon_desc_init_data = {max_width = 1920, max_height = 1080, bit_depth = EB_EIGHT_BIT, color_format = EB_YUV420, buffer_enable_mask = 7, rest_units_per_tile = 21838608, left_padding = 96, right_padding = 96, top_padding = 96, bot_padding = 96, split_mode = 0 '\000', mfmv = 127 '\177', is_16bit_pipeline = 0 '\000', enc_mode = ENC_M0, sb_total_count = 21838608}
        is_16bit = 0 '\000'
        last_frame_filter_level = {0, 0, 0, 0}
        use_qp_for_chroma = false
        scs = 0x7fffd41c56d0
        frm_hdr = 0x7fffd496cb10
        lf = 0x7fffd496cf3c
        sharpness_val = 1
        tune = 2 '\002'
#7  0x00007fffeb04e0d3 in svt_aom_dlf_kernel (input_ptr=0x7fff212a8820) at /home/vi/src/svt-av1-psy/Source/Lib/Codec/dlf_process.c:104
        recon_buffer = 0x7fff4bf389a0
        ppcs = 0x7fffd4967570
        dlf_enable_flag = 1 '\001'
        is_16bit = 0 '\000'
        tg_count = 1
        segment_index = 6
        thread_ctx = 0x7fff212a8820
        context_ptr = 0x7fff212b6560
        pcs = 0x7fff660714b0
        scs = 0x7fffd41c56d0
        enc_dec_results_wrapper = 0x7fff20d37910
        enc_dec_results = 0x7fff65e5d150
        dlf_results_wrapper = 0x7fff20d3ea80
        dlf_results = 0x7fff20d3eac0
#8  0x00007ffff4ea8134 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140733215229632, -1070768396530321341, -128, 0, 140736974846752, 140733214179328, 1071257801744480323, 1070788011622037571}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007ffff4f287dc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
No locals.

The text was updated successfully, but these errors were encountered:

BlueSwordM · 2024-12-25T19:14:17Z

Could you test this with 10-bit? I'd be interested to know if it's reproduceable with that, because I can't seem to do so on my end with 10-bit. Will test with 8-bit soon.

vi · 2024-12-25T20:48:06Z

With -pix_fmt yuv420p10le it encoded properly.

igorbaryshev · 2024-12-26T12:48:50Z

I've been getting a random crash a couple times at different points in the beginning of encode process for the same 10-bit video, but then it worked normally and finished all 4+ hours of encode 🤔
Unfortunately, StaxRip didn't save the log for those, but it mentioned "access violation" or something.
I was trying out the new --enable-tf 2 and --psy-rd 6.0 options

vi · 2024-12-26T14:15:47Z

Why not just try 8-bit video, where the crash is reproducible for me?

Boulder08 · 2024-12-26T14:23:24Z

I think there is something in psy-rd and/or Clang. If I use this command line to test the encoder, it works with the GCC build but adding --psy-rd 1.0 makes it crash at the very beginning with both GCC and Clang. 10-bit source.

avs2yuv64.exe -no-mt "startrek_1440p_2.avs" - | C:\svt-av1\svtav1encapp.exe --preset 2 --lp 0 --scm 0 --enable-qm 1 --keyint 10s --film-grain-denoise 0 --sharpness 3 --tile-columns 1 --tile-rows 0 --enable-tf 2 --enable-restoration 1 --enable-cdef 0 --qm-min 2 --qm-max 15 --tune 3 --hierarchical-levels 4 --qp-scale-compress-strength 2 --frame-luma-bias 7 --noise-norm-strength 4 --color-primaries 9 --transfer-characteristics 16 --matrix-coefficients 9 --use-fixed-qindex-offsets 2 --key-frame-qindex-offset 0 --key-frame-chroma-qindex-offset -32 --qindex-offsets [0,0,0,0,0,0] --chroma-qindex-offsets [-32,-32,-32,-32,-32,-32] --chroma-qm-min 10 --chroma-qm-max 15 --film-grain 16 --variance-octile 5 --crf 16 -b NUL -i -

vi · 2024-12-26T14:31:09Z

Here is repro command line I minimized (without the use of real video):

ffmpeg  -f lavfi -i testsrc2=size=1920x1080 -c:v libx264 -c:v libx264 -preset ultrafast -crf 8 -f matroska - | /opt/ffmpeg/bin/ffmpeg  -i - -c:v libsvtav1 -svtav1-params preset=2:crf=70,scm=1 -f null null

It doesn't reproduce when I use testsrc2 directly, without first going through H.264 for some reason.

Boulder08 · 2024-12-26T14:32:30Z

Now it seems completely random, removing psy-rd suddenly doesn't help with GCC either with the exact same clip.

vi · 2024-12-26T14:40:22Z

For some reason, adding | ffmpeg -i - -f yuv4mpegpipe - | in the middle (i.e. decoding H.264 in a separate FFmpeg instance instead of the one which uses svt-av1-psy) makes it non-reproducible again.

Boulder08 · 2024-12-26T14:56:04Z

I compiled a debug build, but it refuses to crash :(

vi · 2024-12-26T15:16:43Z

Maybe the problem is somehow related to FFmpeg.

Now it assumes ABI compatibility with svt-av1 or previous version of svt-av1-psy. Maybe I'll try to re-compile it.

Boulder08 · 2024-12-26T15:22:10Z

Nope, the crashes happen on the regular Windows command line application as well, also for others.

Boulder08 · 2024-12-26T17:40:07Z

I can confirm that 129faba is the one where the crashes start.

igorbaryshev · 2024-12-26T23:18:34Z

Looks like 8-bit encodes are really messed up, compared to 10-bit, where I've only got crashes on very long encodes.
On me 8-bit crashes after a few hundred frames, sometimes even less than hundred, it's random.
This is the error I get when encoding 8-bit in StaxRip:

C:\tools\encoding\staxrip\Apps\Support\avs2pipemod\avs2pipemod64.exe -dll=C:\tools\encoding\staxrip\Apps\FrameServer\AviSynth\AviSynth.dll -y4mp Z:\encodes\asdf_temp\asdfd.avs | C:\tools\encoding\staxrip\Apps\Encoders\SvtAv1EncApp\SvtAv1EncApp.exe --input - --output Z:\encodes\asdf_temp\asdfd_out.ivf --width 1920 --height 1080 --frames 184066 --rc 0 --progress 3 --preset 6

Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1-PSY Encoder Lib v2.3.0-A-dirty
Svt[info]: SVT [build]  :	Visual Studio 2022 64 bit
Svt[info]: LIB Build date: Dec 26 2024 03:16:22
Svt[info]: -------------------------------------------
avs2pipemod[info]: writing 184066 frames of 25/1 fps, 1920x1080,
                   sar 0:0, YUV-420-planar-8bit progressive video.
Svt[info]: Level of Parallelism: 5
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile	tier (auto)	level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator 		: 1920 / 1080 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format 					: 8 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct 					: 6 / SSIM / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type 			: 321 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor 					: CRF / 35.00 
Svt[info]: SVT [config]: AQ mode / variance boost strength / octile / curve 		: 2 / 2 / 6 / regular
Svt[info]: SVT [config]: Sharpness / QP scale compress strength / Frame low-luma bias 	: 1 / 1 / 0
Svt[info]: SVT [config]: Temporal Filtering Strength 					: 1
Svt[info]: SVT [config]: Keyframe TF Strength 						: 1
Svt[info]: -------------------------------------------
Encoding          
avs2pipemod[info]: total elapsed time is 4.532 sec.
avs2pipemod[error]: only wrote 270 of 184066 frames.


Video encoding returned exit code: -1073741819 (0xC0000005)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It's unclear what this exit code means, in case it's
a Windows system error then it possibly means:

# for decimal -1073741819 / hex 0xc0000005
  STATUS_ACCESS_VIOLATION                                        ntstatus.h   
# The instruction at 0x%p referenced memory at 0x%p. The
# memory could not be %s.
# as an HRESULT: Severity: FAILURE (1), FACILITY_NULL (0x0), Code 0x5
# for decimal 5 / hex 0x5
  ERROR_ACCESS_DENIED                                            winerror.h   
# Access is denied.

vi · 2024-12-27T07:04:39Z

I get the crash only for preset 4 and lower, always when processing the second frame, in debug builds as well as release ones.
Significantly changing video resolution (1920x1080) typically makes it not reproducible.

Do you need a valgrind / sanitizer run?

BlueSwordM · 2024-12-27T18:03:31Z

I get the crash only for preset 4 and lower, always when processing the second frame, in debug builds as well as release ones. Significantly changing video resolution (1920x1080) typically makes it not reproducible.

Do you need a valgrind / sanitizer run?

Yeah, I think that's our best bet.

vi · 2024-12-27T18:16:37Z

Note that a backtrace is already attached in the issue description.

Do you need a full core dump?

BlueSwordM · 2024-12-27T18:23:38Z

Note that a backtrace is already attached in the issue description.

Do you need a full core dump?

Oh no need then. Sorry, it's just that I've been a bit overwhelmed with all of this, considering that the release was done during a holiday.

vi · 2024-12-27T18:27:08Z

Here is -fsanitizer=address run:

$ ffmpeg -v panic  -f lavfi -i testsrc2=size=1080x1920 -c:v libx264 -preset ultrafast -crf 8 -f matroska - | LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.8.0.0 /opt/ffmpeg/bin/ffmpeg  -i - -c:v libsvtav1 -svtav1-params preset=4:crf=40 -f null null
ffmpeg version n7.1-11-g5c59d97e8a Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-libass --enable-frei0r --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --disable-librtmp --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-ffplay --disable-vdpau --enable-libx265 --prefix=/opt/ffmpeg/ --enable-opencl --enable-rpath --enable-libsvtav1
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, matroska,webm, from 'fd:':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1080x1920 [SAR 1:1 DAR 9:16], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> av1 (libsvtav1))
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:       SVT-AV1-PSY Encoder Lib v2.3.0-A-dirty
Svt[info]: SVT [build]  :       GCC 12.2.0       64 bit
Svt[info]: LIB Build date: Dec 27 2024 19:22:13
Svt[info]: -------------------------------------------
Svt[info]: Level of Parallelism: 5
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto)     level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator               : 1080 / 1920 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format                                       : 8 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                                    : 4 / SSIM / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type                      : 321 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor                                         : CRF / 40.00
Svt[info]: SVT [config]: AQ mode / variance boost strength / octile / curve             : 2 / 2 / 6 / regular
Svt[info]: SVT [config]: Sharpness / QP scale compress strength / Frame low-luma bias   : 1 / 1 / 0
Svt[info]: SVT [config]: Temporal Filtering Strength                                    : 1
Svt[info]: SVT [config]: Keyframe TF Strength                                           : 1
Svt[info]: -------------------------------------------
SvtMalloc[info]: SVT Memory Usage:
SvtMalloc[info]:     total allocated memory:       2.27 GB
SvtMalloc[info]:         malloced memory:          975.48 MB
SvtMalloc[info]:         callocated memory:        40.95 MB
SvtMalloc[info]:         allocated aligned memory: 1.28 GB
SvtMalloc[info]:     mutex count: 26265
SvtMalloc[info]:     semaphore count: 681
SvtMalloc[info]:     thread count: 74
SvtMalloc[info]:     hash table fulless: 0.132491, hash bucket is healthy
SvtMalloc[info]: top 10 malloced memory locations:
SvtMalloc[info]: (270.94 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/coding_unit.c:84
SvtMalloc[info]: (152.80 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:114
SvtMalloc[info]: (125.51 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:115
SvtMalloc[info]: (91.58 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/bitstream_unit.c:36
SvtMalloc[info]: (87.66 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:1075
SvtMalloc[info]: (73.96 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:1528
SvtMalloc[info]: (41.09 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:1531
SvtMalloc[info]: (15.94 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:1047
SvtMalloc[info]: (13.01 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:571
SvtMalloc[info]: (12.12 MB): /home/vi/src/svt-av1-psy/Source/Lib/Codec/pcs.c:1312
Output #0, null, to 'null':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: av1, yuv420p(tv, progressive), 1080x1920 [SAR 1:1 DAR 9:16], q=2-31, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.19.100 libsvtav1
=================================================================peed=N/A
==24305==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x621007929510 at pc 0x7f44c7294c97 bp 0x7f444ef979a0 sp 0x7f444ef97998
READ of size 1 at 0x621007929510 thread T86
    #0 0x7f44c7294c96 in svt_sa8d_8x8 /home/vi/src/svt-av1-psy/Source/Lib/Codec/psy_rd.c:57
    #1 0x7f44c729641e in svt_psy_distortion /home/vi/src/svt-av1-psy/Source/Lib/Codec/psy_rd.c:137
    #2 0x7f44c729805c in get_svt_psy_full_dist /home/vi/src/svt-av1-psy/Source/Lib/Codec/psy_rd.c:272
    #3 0x7f44c723b991 in tx_type_search /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:4797
    #4 0x7f44c72439bb in perform_tx_partitioning /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:5307
    #5 0x7f44c7257c7b in full_loop_core /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:6687
    #6 0x7f44c725bf04 in md_stage_3 /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:7007
    #7 0x7f44c7283c50 in md_encode_block /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:9461
    #8 0x7f44c728e8a0 in process_block /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:10382
    #9 0x7f44c72942cb in svt_aom_mode_decision_sb /home/vi/src/svt-av1-psy/Source/Lib/Codec/product_coding_loop.c:10795
    #10 0x7f44c6ec4267 in svt_aom_mode_decision_kernel /home/vi/src/svt-av1-psy/Source/Lib/Codec/enc_dec_process.c:3552
    #11 0x7f44d1aa8133 in start_thread nptl/pthread_create.c:442
    #12 0x7f44d1b287db in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x621007929510 is located 16 bytes to the right of 4096-byte region [0x621007928500,0x621007929500)
allocated by thread T15 (vf#0:0) here:
    #0 0x7f44d4cb7ffd in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f44c7188c52 in svt_picture_buffer_desc_ctor /home/vi/src/svt-av1-psy/Source/Lib/Codec/pic_buffer_desc.c:270
    #2 0x7f44c70b4d87 in svt_aom_mode_decision_cand_bf_ctor /home/vi/src/svt-av1-psy/Source/Lib/Codec/mode_decision.c:650
    #3 0x7f44c710e2b6 in svt_aom_mode_decision_context_ctor /home/vi/src/svt-av1-psy/Source/Lib/Codec/md_process.c:456
    #4 0x7f44c6e98b41 in svt_aom_enc_dec_context_ctor /home/vi/src/svt-av1-psy/Source/Lib/Codec/enc_dec_process.c:122
    #5 0x7f44c6d93205 in svt_av1_enc_init /home/vi/src/svt-av1-psy/Source/Lib/Globals/enc_handle.c:2579
    #6 0x7f44d2e95b37  (/opt/ffmpeg//lib/libavcodec.so.61+0x95b37)

Thread T86 created by T15 (vf#0:0) here:
    #0 0x7f44d4c49726 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x7f44c73539d2 in svt_create_thread /home/vi/src/svt-av1-psy/Source/Lib/Codec/svt_threads.c:166
    #2 0x7f44c6d98134 in svt_av1_enc_init /home/vi/src/svt-av1-psy/Source/Lib/Globals/enc_handle.c:2694
    #3 0x7f44d2e95b37  (/opt/ffmpeg//lib/libavcodec.so.61+0x95b37)

Thread T15 (vf#0:0) created by T0 here:
    #0 0x7f44d4c49726 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x55c56e09d81b  (/opt/ffmpeg/bin/ffmpeg+0x2d81b)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/vi/src/svt-av1-psy/Source/Lib/Codec/psy_rd.c:57 in svt_sa8d_8x8
Shadow bytes around the buggy address:
  0x0c4280f1d250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4280f1d260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4280f1d270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4280f1d280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4280f1d290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c4280f1d2a0: fa fa[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4280f1d2b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4280f1d2c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4280f1d2d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4280f1d2e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4280f1d2f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==24305==ABORTING

vi · 2024-12-27T19:31:04Z

Most commits I tried fail the -fsanitizer=address test on preset 7 or lower.

Starting with with bd70bf1 as good commit (that passes the sanitizer), git bisect points to 9d6572d as the first commit where sanitizer starts detecting errors (on preset 3 and below).

But previous commit 0d32981 still fails with preset 0 with large video size, as well as bd70bf1.

Started again (with preset=0 1080p ssim) based on 2aeeb4f as passing commit. Now it pointed to commit 3a7b34c.

$ ffmpeg -v panic  -f lavfi -i testsrc2=size=1920x1080 -c:v libx264 -preset ultrafast -crf 8 -f matroska - | LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.8.0.0 /opt/ffmpeg/bin/ffmpeg  -i - -c:v libsvtav1 -svtav1-params preset=0:tune=2 -f null null
ffmpeg version n7.1-11-g5c59d97e8a Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-libass --enable-frei0r --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --disable-librtmp --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-ffplay --disable-vdpau --enable-libx265 --prefix=/opt/ffmpeg/ --enable-opencl --enable-rpath --enable-libsvtav1
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, matroska,webm, from 'fd:':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> av1 (libsvtav1))
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:       SVT-AV1 Encoder Lib 3a7b34c7
Svt[info]: SVT [build]  :       GCC 12.2.0       64 bit
Svt[info]: LIB Build date: Dec 27 2024 21:50:31
Svt[info]: -------------------------------------------
Svt[warn]: Film grain denoise apply signal is going to be ignored when film grain is off.
Svt[warn]: Instance 1: tune ssim (2) is supported for testing and debugging purposes.This configuration should not be used for any benchmarking analysis at this stage
Svt[info]: Number of logical cores available: 12
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto)     level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator               : 1920 / 1080 / 25 / 1
Svt[info]: SVT [config]: bit-depth / color format                                       : 8 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                                    : 0 / SSIM / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type                      : 161 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor                                         : CRF / 35
Svt[info]: SVT [config]: AQ mode / variance boost                                       : 2 / 0
Svt[info]: -------------------------------------------
SvtMalloc[info]: SVT Memory Usage:
SvtMalloc[info]:     total allocated memory:       4.50 GB
SvtMalloc[info]:         malloced memory:          1.23 GB
SvtMalloc[info]:         callocated memory:        69.38 MB
SvtMalloc[info]:         allocated aligned memory: 3.20 GB
SvtMalloc[info]:     mutex count: 25155
SvtMalloc[info]:     semaphore count: 1743
SvtMalloc[info]:     thread count: 412
SvtMalloc[info]:     hash table fulless: 0.122404, hash bucket is healthy
SvtMalloc[info]: top 10 malloced memory locations:
SvtMalloc[info]: (286.88 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbCodingUnit.c:83
SvtMalloc[info]: (152.80 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:114
SvtMalloc[info]: (125.51 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:115
SvtMalloc[info]: (92.81 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:1074
SvtMalloc[info]: (91.58 MB): /home/vi/src/svt-av1-psy/Source/Lib/Common/Codec/EbBitstreamUnit.c:36
SvtMalloc[info]: (91.07 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbModeDecisionProcess.c:280
SvtMalloc[info]: (73.96 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:1527
SvtMalloc[info]: (41.09 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:1530
SvtMalloc[info]: (38.79 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbModeDecisionProcess.c:376
SvtMalloc[info]: (26.98 MB): /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbModeDecisionProcess.c:343
Output #0, null, to 'null':
  Metadata:
    encoder         : Lavf61.7.100
  Stream #0:0: Video: av1, yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.19.100 libsvtav1
=================================================================speed=   0x
==6023==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61d0000ad078 at pc 0x7ff65eece0ce bp 0x7ff5acf2d7a0 sp 0x7ff5acf2d798
READ of size 4 at 0x61d0000ad078 thread T307
    #0 0x7ff65eece0cd in get_sb128_me_data /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EncModeConfig.c:2931
    #1 0x7ff65eeec6c1 in svt_aom_set_depth_ctrls /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EncModeConfig.c:6679
    #2 0x7ff65eefc26a in svt_aom_sig_deriv_enc_dec_common /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EncModeConfig.c:8281
    #3 0x7ff65ea1013c in svt_aom_mode_decision_kernel /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbEncDecProcess.c:3432
    #4 0x7ff668ea8133 in start_thread nptl/pthread_create.c:442
    #5 0x7ff668f287db in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x61d0000ad078 is located 0 bytes to the right of 2040-byte region [0x61d0000ac880,0x61d0000ad078)
allocated by thread T15 (vf#0:0) here:
    #0 0x7ff66c0b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7ff65eb9f680 in picture_parent_control_set_ctor /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:1335
    #2 0x7ff65eba75e5 in svt_aom_picture_parent_control_set_creator /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EbPictureControlSet.c:1686
    #3 0x7ff65e205f0e in svt_object_wrapper_ctor /home/vi/src/svt-av1-psy/Source/Lib/Common/Codec/EbSystemResourceManager.c:400
    #4 0x7ff65e20694a in svt_system_resource_ctor /home/vi/src/svt-av1-psy/Source/Lib/Common/Codec/EbSystemResourceManager.c:452
    #5 0x7ff65e93a162 in svt_av1_enc_init /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Globals/EbEncHandle.c:1632
    #6 0x7ff66a295b37  (/opt/ffmpeg//lib/libavcodec.so.61+0x95b37)

Thread T307 created by T15 (vf#0:0) here:
    #0 0x7ff66c049726 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x7ff65e208764 in svt_create_thread /home/vi/src/svt-av1-psy/Source/Lib/Common/Codec/EbThreads.c:166
    #2 0x7ff65e950023 in svt_av1_enc_init /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Globals/EbEncHandle.c:2367
    #3 0x7ff66a295b37  (/opt/ffmpeg//lib/libavcodec.so.61+0x95b37)

Thread T15 (vf#0:0) created by T0 here:
    #0 0x7ff66c049726 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x56488488381b  (/opt/ffmpeg/bin/ffmpeg+0x2d81b)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/vi/src/svt-av1-psy/Source/Lib/Encoder/Codec/EncModeConfig.c:2931 in get_sb128_me_data
Shadow bytes around the buggy address:
  0x0c3a8000d9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3a8000d9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3a8000d9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3a8000d9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3a8000d9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c3a8000da00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[fa]
  0x0c3a8000da10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8000da20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8000da30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8000da40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8000da50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==6023==ABORTING

Maybe there is a lot of undefined behaviour lurking in the project?
Maybe ASan check should be added to CI?

igorbaryshev · 2024-12-27T20:07:12Z

Injecting mimalloc fixes the crash for me

vi · 2024-12-27T21:05:11Z

LD_PRELOADing libmimalloc.so.2.0 indeed helps mitigating things, but it does not fool ASan.

igorbaryshev · 2024-12-28T04:27:44Z

with statically linked mimalloc it still crashed

gitoss · 2024-12-28T18:38:04Z

Injecting mimalloc fixes the crash for me

Alas, it doesn't fix it for me - only lowering --lp does.

vi added the bug Something isn't working label Dec 25, 2024

someonenameguy mentioned this issue Jan 12, 2025

For Encoder Preset lower than 8. Job Fails: Worker process exited! Worker process exit was not expected. Nj0be/HandBrake-SVT-AV1-PSY#14

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] regression from 916cabd50bba to v2.3.0-A: crash #114

[BUG] regression from 916cabd50bba to v2.3.0-A: crash #114

vi commented Dec 25, 2024 •

edited by BlueSwordM

Loading

BlueSwordM commented Dec 25, 2024 •

edited

Loading

vi commented Dec 25, 2024

igorbaryshev commented Dec 26, 2024 •

edited

Loading

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024 •

edited

Loading

vi commented Dec 26, 2024 •

edited

Loading

Boulder08 commented Dec 26, 2024

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

igorbaryshev commented Dec 26, 2024

vi commented Dec 27, 2024 •

edited

Loading

BlueSwordM commented Dec 27, 2024

vi commented Dec 27, 2024

BlueSwordM commented Dec 27, 2024 •

edited

Loading

vi commented Dec 27, 2024

vi commented Dec 27, 2024 •

edited

Loading

igorbaryshev commented Dec 27, 2024

vi commented Dec 27, 2024

igorbaryshev commented Dec 28, 2024

gitoss commented Dec 28, 2024 •

edited

Loading

[BUG] regression from 916cabd50bba to v2.3.0-A: crash #114

[BUG] regression from 916cabd50bba to v2.3.0-A: crash #114

Comments

vi commented Dec 25, 2024 • edited by BlueSwordM Loading

BlueSwordM commented Dec 25, 2024 • edited Loading

vi commented Dec 25, 2024

igorbaryshev commented Dec 26, 2024 • edited Loading

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024 • edited Loading

vi commented Dec 26, 2024 • edited Loading

Boulder08 commented Dec 26, 2024

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

vi commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

Boulder08 commented Dec 26, 2024

igorbaryshev commented Dec 26, 2024

vi commented Dec 27, 2024 • edited Loading

BlueSwordM commented Dec 27, 2024

vi commented Dec 27, 2024

BlueSwordM commented Dec 27, 2024 • edited Loading

vi commented Dec 27, 2024

vi commented Dec 27, 2024 • edited Loading

igorbaryshev commented Dec 27, 2024

vi commented Dec 27, 2024

igorbaryshev commented Dec 28, 2024

gitoss commented Dec 28, 2024 • edited Loading

vi commented Dec 25, 2024 •

edited by BlueSwordM

Loading

BlueSwordM commented Dec 25, 2024 •

edited

Loading

igorbaryshev commented Dec 26, 2024 •

edited

Loading

Boulder08 commented Dec 26, 2024 •

edited

Loading

vi commented Dec 26, 2024 •

edited

Loading

vi commented Dec 27, 2024 •

edited

Loading

BlueSwordM commented Dec 27, 2024 •

edited

Loading

vi commented Dec 27, 2024 •

edited

Loading

gitoss commented Dec 28, 2024 •

edited

Loading