validate with generics and arrays

[NinjaKinshasa]

Hello marcj,

Thank you for your work, I recently found deepkit and I'm trying to learn how to use it.

My interest is in the runtime validation provided by deepkit/type to ensure that my SQL queries are returning objects that matches with my typescript interfaces.

Unfortunately, I can't get it to work, because the validate function does not seem to handle correctly array type : it only validates the first item of the array.
Also, the validate function does not work with generic types.

import { validate  } from '@deepkit/type';

interface User {
    id: number,
    wrongKey1: string, // this key does not exist
};

const rows = [
    {id: 1, login: "john"},
    {id: 2, login: "mark"},
    {id: 3, login: "mike"}
]

const withInterface =  () => {
    console.log(validate<User[]>(rows));
}

const withGeneric =  <T>() => {
    console.log(validate<T[]>(rows));

}

withInterface();
/* 
Prints [
  ValidationErrorItem {
    path: '0.wrongKey1',
    code: 'type',
    message: 'Not a string',
    value: undefined
  }
]

It only check the first item.
*/

withGeneric<User>();
/*
 Prints [] (empty array)

It does not detect the error.
*/

Even if it is hacky and I don't like it, I can loop over the array and call validate on each item.
However, I do not see any workaround to deal with the generic type issue.

My initial idea was to do something like this :

async query<T>(sql: string, values: any[] = []): Promise<T[]> {
    const rows = await database.query(sql, values);

   const errors = validate<T[]>(rows);
    if (errors.length) {
      throw Error("...");
    }
   
  return cast<T[]>(rows);
}

But this does not work because validate always returns an empty array when using generic types.
With this exemple, the cast returns an array of undefined values wether or not the interface is matching the objects.
I also tried to use assert but it works like validate, which means that it doesn't detect the errors when using generics.

What am I doing wrong here ? Thank you for you help

[marcj]

function does not seem to handle correctly array type : it only validates the first item of the array

That's correct. The code of the array validator breaks after the first invalid entry was found.

the validate function does not work with generic types.

It does work with generics, but not out of the box automatically. In order to not have runtime overhead for each and every generic type argument, you have to explicitly tell Deepkit to embed the generic type into runtime. Otherwise the overhead would be too big and runtime code too slow.

async function query<T>(sql: string, values: any[] = [], type?: ReceiveType<T>): Promise<T[]> {
    type = resolveReceiveType(type);
    const arrayType: TypeArray = { kind: ReflectionKind.array, type };

    const rows = await database.query(sql, values);

   const errors = validate<T[]>(rows, arrayType);
    if (errors.length) {
      throw Error("...");
    }
   
  return cast<T[]>(rows, undefined, undefined, undefined, arrayType);
}

or

async function query<T>(sql: string, values: any[] = [], type?: ReceiveType<T>): Promise<T[]> {
    type = resolveReceiveType(type);
    type ArrayType = InlineRuntimeType<typeof type>[];

    const rows = await database.query(sql, values);

   const errors = validate<ArrayType>(rows, arrayType);
    if (errors.length) {
      throw Error("...");
    }
   
  return cast<ArrayType>(rows, undefined, undefined, undefined, arrayType);
}

or

async function query<T>(sql: string, values: any[] = [], type?: ReceiveType<T>): Promise<T[]> {
    type = resolveReceiveType(type);
    type F = InlineRuntimeType<typeof type>;

    const rows = await database.query(sql, values);

   const errors = validate<F[]>(rows, arrayType);
    if (errors.length) {
      throw Error("...");
    }
   
  return cast<F[]>(rows, undefined, undefined, undefined, arrayType);
}

[NinjaKinshasa]

Thank you for your quick and useful answer.

That's correct. The code of the array validator breaks after the first invalid entry was found.

May I ask why ? To me, the error path being 0.wrongKey1 suggests that the others indexes (1, 2 etc) do not generate errors.

It does work with generics, but not out of the box automatically. In order to not have runtime overhead for each and every generic type argument, you have to explicitly tell Deepkit to embed the generic type into runtime. Otherwise the overhead would be too big and runtime code too slow.

This is clear thank you. However, to me, it seems to be some "noise" in the code you provided :

async function query<T>(sql: string, values: any[] = [], type?: ReceiveType<T>): Promise<T[]> {
The type parameter is infered from the T generic.. It should not be necessary to add a parameter for it.

const errors = validate<T[]>(rows, arrayType);
Same here, why arrayType is necessary as a parameter ?

const arrayType: TypeArray = { kind: ReflectionKind.array, type };
If type is the runtime type, why type[] does not work directly out of the box ?

As a developer who have absolutely zero knowledge of how typing works inside, what I would prefer to do is something like that :

async function query<T>(sql: string, values: any[] = []): Promise<T[]> {
    type dynamicType = resolveRuntimeType<T>();

    const rows = await database.query(sql, values);

   const errors = validate<dynamicType[]>(rows, arrayType);

    if (errors.length) {
      throw Error("...");
    }
   
  return ...;
}

I am not sure if a function can return a type or an interface like that, and I have no idea of the internal complexity of this, but as a developer, this would feel way more natural to do something straightforward like "get runtime type" then use it like any other type, not like an object.

[marcj]

@NinjaKinshasa I agree. Having to deal with type from ReceiveType here is cumbersome. I've improved it in 4d24c8b
and now it's possible to have something like

async function query<T>(sql: string, values: any[] = [], type?: ReceiveType<T>): Promise<T[]> {
    const rows = await database.query(sql, values);

   const errors = validate<T[]>(rows);
    if (errors.length) {
      throw Error("...");
    }
   
  return cast<T[]>(rows);
}

A lot less boilerplate code. The only necessary marker is to use type?: ReceiveType<T> so that the compiler knows this function is special and wants to receive type arguments in runtime.