Segmentation fault when I call add_item, version: v1.16.3

[longjiquan]

Hello, I tried to create an annoy index with DotProduct. But when I called add_item, the program crashed due to segment fault. Below are my test codes:

#include <iostream>
#include <memory>
#include <random>
#include <vector>

#include "../src/annoylib.h"
#include "../src/kissrandom.h"

void dot() {
  size_t dim = 512;
  size_t n_trees = 10;
  auto index =
      std::make_shared<AnnoyIndex<int64_t, float, DotProduct, Kiss64Random>>(
          dim);

  std::vector<float> fdata;
  auto fp = fopen("/tmp/annoy-vectors.dat", "rb");
  if (fp == nullptr) {
    exit(-1);
  }

  auto buf = new float;
  while (1 == fread(buf, sizeof(float), 1, fp)) {
    fdata.push_back(*buf);
  }
  delete buf;
  fclose(fp);

  std::cout << fdata.size() << std::endl;
  if (fdata.size() % dim != 0) {
    exit(-1);
  }

  for (size_t i = 0; i < fdata.size() / dim; i++) {
    index->add_item(i, fdata.data() + i * dim);
  }

  index->build(n_trees);
}

int main(int argc, char* argv) {
  dot();

  return 0;
}

Annoy Version: v1.16.3
Dim: 512
Num of Vectors: 25097
Let me know if you need the file /tmp/annoy-vectors.dat.

Call stack:

(annoy1.16.3) ljq@spark-slave~/work/annoy/examples(v1.16.3)$ gdb ./dot_test
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./dot_test...done.
(gdb) r
Starting program: /home/ljq/work/annoy/examples/dot_test 
12849664

Program received signal SIGSEGV, Segmentation fault.
0x00005555555591cc in DotProduct::create_split<long, float, Kiss64Random> (
    nodes=std::vector of length 2544, capacity 4096 = {...}, f=512, s=2076, 
    random=..., n=0x7fffff7ffae0) at ../src/annoylib.h:544
544	    DotProduct::zero_value(p); 
(gdb) bt
#0  0x00005555555591cc in DotProduct::create_split<long, float, Kiss64Random> (
    nodes=std::vector of length 2544, capacity 4096 = {...}, f=512, s=2076, 
    random=..., n=0x7fffff7ffae0) at ../src/annoylib.h:544
#1  0x0000555555557e4b in AnnoyIndex<long, float, DotProduct, Kiss64Random>::_make_tree (this=0x55555577ae80, 
    indices=std::vector of length 2544, capacity 4096 = {...}, is_root=false)
    at ../src/annoylib.h:1199
#2  0x00005555555581f7 in AnnoyIndex<long, float, DotProduct, Kiss64Random>::_make_tree (this=0x55555577ae80, 
    indices=std::vector of length 2545, capacity 4096 = {...}, is_root=false)
    at ../src/annoylib.h:1239
#3  0x00005555555581f7 in AnnoyIndex<long, float, DotProduct, Kiss64Random>::_make_tree (this=0x55555577ae80, 
    indices=std::vector of length 2546, capacity 4096 = {...}, is_root=false)
    at ../src/annoylib.h:1239
#4  0x00005555555581f7 in AnnoyIndex<long, float, DotProduct, Kiss64Random>::_make_tree (this=0x55555577ae80, 
    indices=std::vector of length 2547, capacity 4096 = {...}, is_root=false)
    at ../src/annoylib.h:1239
#5  0x00005555555581f7 in AnnoyIndex<long, float, DotProduct, Kiss64Random>::_make_tree (this=0x55555577ae80, 
    indices=std::vector of length 2548, capacity 4096 = {...}, is_root=false)
    at ../src/annoylib.h:1239
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) 

I will appreciate your quick reply.

[longjiquan]

The data can be generated by:

import struct
import numpy as np

def float_to_bytes(f):
    return struct.pack('f', f)

dim = 512
num = 25097
vectors = np.random.random([num, dim]).tolist()

with open('/tmp/annoy-vectors.dat', 'wb') as file:
    for vector in vectors:
        for f in vector:
            file.write(float_to_bytes(f))