Grammars cannot have an “id” field

[tushar5526]

What is the expected behaviour here ? @ChakshuGautam

Should we override the default id field that is being created or raise an error saying columns can not have "id" as name.

For example, following grammar will error out.

vPK,Index
string,string
id,category_name

Error

tusharsamagra@Tushars-MacBook-Pro c-qube % yarn cli ingest
yarn run v1.22.19
warning ../../../../package.json: No license field
$ ts-node src/console.ts ingest
[Nest] 6942  - 09/04/2023, 1:38:39 PM     LOG [NestFactory] Starting Nest application...
[Nest] 6942  - 09/04/2023, 1:38:39 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +20ms
[Nest] 6942  - 09/04/2023, 1:38:39 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +1ms
[Nest] 6942  - 09/04/2023, 1:38:39 PM     LOG [InstanceLoader] AppModule dependencies initialized +0ms
┌  Starting Ingestion Process
│
◇  ✅ 1. The Data has been Nuked
│
◇  ✅ 2. Config parsing completed
│
○  🚧 3. Processing Dimensions
categorypgi
{
  "name": "categorypgi",
  "description": "",
  "type": "dynamic",
  "storage": {
    "indexes": [
      "name"
    ],
    "primaryId": "id",
    "retention": null,
    "bucket_size": null
  },
  "schema": {
    "title": "categorypgi",
    "psql_schema": "dimensions",
    "properties": {
      "id": {
        "type": "string",
        "unique": true
      },
      "category_name": {
        "type": "string",
        "unique": true
      }
    },
    "indexes": [
      {
        "columns": [
          [
            "category_name"
          ]
        ]
      }
    ]
  }
}
{
  createQuery: 'CREATE TABLE dimensions.categorypgi (id SERIAL PRIMARY KEY, id VARCHAR UNIQUE, category_name VARCHAR UNIQUE);'
}
{
  indexQuery: [
    'CREATE INDEX categorypgi_category_name_idx ON dimensions.categorypgi (category_name);'
  ]
}
categorypgi
CREATE INDEX categorypgi_category_name_idx ON dimensions.categorypgi (category_name);
PrismaClientKnownRequestError: 
Invalid `prisma.$queryRaw()` invocation:


Raw query failed. Code: `42P01`. Message: `relation "dimensions.categorypgi" does not exist`
    at RequestHandler.handleRequestError (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:35024:13)
    at RequestHandler.handleAndLogRequestError (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:34996:12)
    at RequestHandler.request (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:34991:12)
    at async Proxy._request (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:36082:16)
    at async DimensionService.createDimension (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/services/dimension/dimension.service.ts:105:7)
    at async CsvAdapterService.ingest (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/services/csv-adapter/csv-adapter.service.ts:130:9)
    at async Object.handler (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/console.ts:24:7) {
  code: 'P2010',
  clientVersion: '4.8.1',
  meta: {
    code: '42P01',
    message: 'relation "dimensions.categorypgi" does not exist'
  },
  batchRequestIdx: undefined
}
After categorypgi 7
INSERT INTO dimensions.categorypgi (category_name) VALUES ('Outcome'), ('Effective Classroom Transaction'), ('Infrastructure Facilities Student Entitlements'), ('School Safety and Child Protection'), ('Digital Learning'), ('Governance Processes'), ('Overall');
[
  { id: 0, category_id: 1, category_name: 'Outcome' },
  {
    id: 1,
    category_id: 2,
    category_name: 'Effective Classroom Transaction'
  },
  {
    id: 2,
    category_id: 3,
    category_name: 'Infrastructure Facilities Student Entitlements'
  },
  {
    id: 3,
    category_id: 4,
    category_name: 'School Safety and Child Protection'
  },
  { id: 4, category_id: 5, category_name: 'Digital Learning' },
  { id: 5, category_id: 6, category_name: 'Governance Processes' },
  { id: 6, category_id: 7, category_name: 'Overall' }
]
categorypgi
PrismaClientKnownRequestError: 
Invalid `prisma.$queryRaw()` invocation:


Raw query failed. Code: `42P01`. Message: `relation "dimensions.categorypgi" does not exist`
    at RequestHandler.handleRequestError (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:35024:13)
    at RequestHandler.handleAndLogRequestError (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:34996:12)
    at RequestHandler.request (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:34991:12)
    at async Proxy._request (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/node_modules/@prisma/client/runtime/index.js:36082:16)
    at async DimensionService.insertBulkDimensionDataV2 (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/services/dimension/dimension.service.ts:175:5)
    at async Promise.all (index 0)
    at async CsvAdapterService.ingest (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/services/csv-adapter/csv-adapter.service.ts:179:5)
    at async Object.handler (/Users/tusharsamagra/projects/cQube-ingestion/impl/c-qube/src/console.ts:24:7) {
  code: 'P2010',
  clientVersion: '4.8.1',
  meta: {
    code: '42P01',
    message: 'relation "dimensions.categorypgi" does not exist'
  },
│
◇  ✅ 3. Dimensions have been ingested
│
◇  ✅ 4. Event Grammars have been ingested
│
◇  ✅ 5. Dataset Grammars have been ingested
│
└  You're all set!

✨  Done in 1.89s.

[tushar5526]

Index,
string,string
id,academicyear

@ChakshuGautam what is the expected behaviour in this case? If we don't have a PK specified but there is a column that is not a PK but has conflicting names ?

[tushar5526]

I think a good approach to make it backwards-compatible as well would be to not allow "id" as column ?