Skip to content

Commit

Permalink
feat: add package management queries and update dependencies
Browse files Browse the repository at this point in the history
  • Loading branch information
Sma1lboy committed Jan 23, 2025
1 parent d5ae48b commit 1bf8193
Show file tree
Hide file tree
Showing 10 changed files with 1,951 additions and 3,723 deletions.
99 changes: 20 additions & 79 deletions backend/README.md
Original file line number Diff line number Diff line change
@@ -1,99 +1,40 @@
<p align="center">
<a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="120" alt="Nest Logo" /></a>
</p>
# Backend Project Setup

[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456
[circleci-url]: https://circleci.com/gh/nestjs/nest
## Installation

<p align="center">A progressive <a href="http://nodejs.org" target="_blank">Node.js</a> framework for building efficient and scalable server-side applications.</p>
<p align="center">
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/v/@nestjs/core.svg" alt="NPM Version" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/l/@nestjs/core.svg" alt="Package License" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/dm/@nestjs/common.svg" alt="NPM Downloads" /></a>
<a href="https://circleci.com/gh/nestjs/nest" target="_blank"><img src="https://img.shields.io/circleci/build/github/nestjs/nest/master" alt="CircleCI" /></a>
<a href="https://coveralls.io/github/nestjs/nest?branch=master" target="_blank"><img src="https://coveralls.io/repos/github/nestjs/nest/badge.svg?branch=master#9" alt="Coverage" /></a>
<a href="https://discord.gg/G7Qnnhy" target="_blank"><img src="https://img.shields.io/badge/discord-online-brightgreen.svg" alt="Discord"/></a>
<a href="https://opencollective.com/nest#backer" target="_blank"><img src="https://opencollective.com/nest/backers/badge.svg" alt="Backers on Open Collective" /></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://opencollective.com/nest/sponsors/badge.svg" alt="Sponsors on Open Collective" /></a>
<a href="https://paypal.me/kamilmysliwiec" target="_blank"><img src="https://img.shields.io/badge/Donate-PayPal-ff3f59.svg" alt="Donate us"/></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://img.shields.io/badge/Support%20us-Open%20Collective-41B883.svg" alt="Support us"></a>
<a href="https://twitter.com/nestframework" target="_blank"><img src="https://img.shields.io/twitter/follow/nestframework.svg?style=social&label=Follow" alt="Follow us on Twitter"></a>
</p>
<!--[![Backers on Open Collective](https://opencollective.com/nest/backers/badge.svg)](https://opencollective.com/nest#backer)
[![Sponsors on Open Collective](https://opencollective.com/nest/sponsors/badge.svg)](https://opencollective.com/nest#sponsor)-->

## Description

[Nest](https://github.com/nestjs/nest) framework TypeScript starter repository.

## Project setup
To install the project dependencies, run the following command:

```bash
$ pnpm install
pnpm install
```

## Compile and run the project

```bash
# development
$ pnpm run start

# watch mode
$ pnpm run start:dev

# production mode
$ pnpm run start:prod
```
## Running the Project

## Run tests
To start the project in development mode, use:

```bash
# unit tests
$ pnpm run test

# e2e tests
$ pnpm run test:e2e

# test coverage
$ pnpm run test:cov
pnpm run dev
```

## Deployment
> **Note:** The `pnpm run dev` command will first build `codefox-common` before starting the backend. If you use `pnpm run start:dev`, it will not build `codefox-common`.
When you're ready to deploy your NestJS application to production, there are some key steps you can take to ensure it runs as efficiently as possible. Check out the [deployment documentation](https://docs.nestjs.com/deployment) for more information.

If you are looking for a cloud-based platform to deploy your NestJS application, check out [Mau](https://mau.nestjs.com), our official platform for deploying NestJS applications on AWS. Mau makes deployment straightforward and fast, requiring just a few simple steps:
For production mode, build and start the project:

```bash
$ pnpm install -g mau
$ mau deploy
pnpm run build
pnpm run start:prod
```

With Mau, you can deploy your application in just a few clicks, allowing you to focus on building features rather than managing infrastructure.

## Resources

Check out a few resources that may come in handy when working with NestJS:

- Visit the [NestJS Documentation](https://docs.nestjs.com) to learn more about the framework.
- For questions and support, please visit our [Discord channel](https://discord.gg/G7Qnnhy).
- To dive deeper and get more hands-on experience, check out our official video [courses](https://courses.nestjs.com/).
- Deploy your application to AWS with the help of [NestJS Mau](https://mau.nestjs.com) in just a few clicks.
- Visualize your application graph and interact with the NestJS application in real-time using [NestJS Devtools](https://devtools.nestjs.com).
- Need help with your project (part-time to full-time)? Check out our official [enterprise support](https://enterprise.nestjs.com).
- To stay in the loop and get updates, follow us on [X](https://x.com/nestframework) and [LinkedIn](https://linkedin.com/company/nestjs).
- Looking for a job, or have a job to offer? Check out our official [Jobs board](https://jobs.nestjs.com).

## Support
## Running Tests

Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support).
To run all tests, use:

## Stay in touch

- Author - [Kamil Myśliwiec](https://twitter.com/kammysliwiec)
- Website - [https://nestjs.com](https://nestjs.com/)
- Twitter - [@nestframework](https://twitter.com/nestframework)
```bash
pnpm run test
```

## License
If you need to run integration tests for the build system, set the `INTEGRATION_TEST` environment variable:

Nest is [MIT licensed](https://github.com/nestjs/nest/blob/master/LICENSE).
```bash
INTEGRATION_TEST=1 npx jest
```
9 changes: 7 additions & 2 deletions backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@
},
"dependencies": {
"@apollo/server": "^4.11.0",
"@babel/parser": "^7.26.5",
"@babel/traverse": "^7.26.5",
"@babel/types": "^7.26.5",
"@huggingface/hub": "latest",
"@huggingface/transformers": "latest",
"@nestjs/apollo": "^12.2.0",
Expand All @@ -45,6 +48,8 @@
"@types/toposort": "^2.0.7",
"axios": "^1.7.7",
"bcrypt": "^5.1.1",
"chromadb": "^1.10.3",
"chromadb-default-embed": "^2.13.2",
"class-validator": "^0.14.1",
"dotenv": "^16.4.7",
"eslint-plugin-unused-imports": "^4.1.4",
Expand All @@ -57,15 +62,15 @@
"lodash": "^4.17.21",
"markdown-to-txt": "^2.0.1",
"normalize-path": "^3.0.0",
"openai": "^4.77.0",
"openai": "^4.78.1",
"p-queue-es5": "^6.0.2",
"pacote": "^21.0.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"sqlite3": "^5.1.7",
"subscriptions-transport-ws": "^0.11.0",
"toposort": "^2.0.2",
"tar": "^7.4.3",
"toposort": "^2.0.2",
"typeorm": "^0.3.20",
"uuid": "^10.0.0"
},
Expand Down
Original file line number Diff line number Diff line change
@@ -1,87 +1,69 @@
/**
* This test is not a integration test now,
* there is some issue with jest integration test setup with chromadb
*/
import { Logger } from '@nestjs/common';
import DependenciesEmbeddingHandler from '../dependencies-embedding-handler';
import DependenciesEmbeddingHandler from '../dependencies-context/dependencies-embedding-handler';

// Initialize a global logger instance
const logger = new Logger('dependencies embed tester');

// Only run integration tests if INTEGRATION_TEST environment variable is set to '1'
const isIntegrationTest = process.env.INTEGRATION_TEST === '1';

if (!isIntegrationTest) {
logger.log(
'Integration tests are skipped. Set INTEGRATION_TEST=1 to run them.',
);
} else {
describe('DependenciesEmbeddingHandler Integration Tests', () => {
// Mock ChromaDB
jest.mock('chromadb', () => ({
ChromaClient: jest.fn().mockImplementation(() => ({
getOrCreateCollection: jest.fn().mockResolvedValue({
add: jest.fn().mockResolvedValue(true),
query: jest.fn().mockResolvedValue({
documents: [
[
JSON.stringify({
name: 'react',
version: '18.2.0',
content: 'React component lifecycle',
}),
],
],
metadatas: [[{ name: 'react', version: '18.2.0' }]],
}),
}),
})),
}));

// Mock Array.isArray
const originalArrayIsArray = Array.isArray;
Array.isArray = function (type: any): boolean {
if (
type?.constructor?.name === 'Float32Array' ||
type?.constructor?.name === 'BigInt64Array'
) {
return true;
}
return originalArrayIsArray(type);
} as typeof Array.isArray;

(isIntegrationTest ? describe : describe.skip)(
'DependenciesEmbeddingHandler Integration Tests',
() => {
let handler: DependenciesEmbeddingHandler;

// Increase the default timeout for integration tests
jest.setTimeout(300000); // 5 minutes

beforeAll(async () => {
logger.log(
'Initializing DependenciesEmbeddingHandler for integration tests...',
);
beforeEach(async () => {
handler = new DependenciesEmbeddingHandler();
// Wait for the handler to initialize
await new Promise((resolve) => setTimeout(resolve, 5000)); // Wait 5 seconds
logger.log('Initialization complete.');
});

afterAll(() => {
logger.log('Integration tests completed.');
await new Promise((resolve) => setTimeout(resolve, 1000));
});

/**
* Integration Test Case: Add Real Packages and Perform a Search
*
* Purpose:
* - To verify that DependenciesEmbeddingHandler can handle real packages by fetching their type definitions,
* generating embeddings, and storing them correctly.
* - To ensure that the search functionality can retrieve relevant packages based on a real query.
*
* Steps:
* 1. Add multiple real npm packages using the addPackage method.
* 2. Perform a search with a query related to one of the added packages.
* 3. Validate that the search results include the relevant package(s) and are correctly ranked.
*/
test('should add real packages and perform a relevant search', async () => {
// Define real packages to add
const packagesToAdd = [
{ name: 'lodash', version: '4.17.21' },
// { name: 'express', version: '4.18.2' },
// { name: 'react', version: '18.2.0' },
// { name: 'typescript', version: '4.9.5' },
{ name: 'react', version: '18.2.0' },
];

logger.log('Adding real packages...');

// Add all packages concurrently
await handler.addPackages(packagesToAdd);

logger.log('Packages added successfully.');

// Define a search query related to one of the packages, e.g., React
const searchQuery = 'React component lifecycle methods';

logger.log('Executing search with query:', searchQuery);

// Perform the search
const results = await handler.searchContext(searchQuery);

logger.log('Search results received.');

// Validate that results are returned
expect(results.length).toBeGreaterThan(0);

// Check that at least one of the top results is related to 'react'
const topResult = results[0];
expect(topResult.name).toBe('react');
expect(topResult.version).toBe('18.2.0');

logger.log('Top search result:', topResult);

// Optionally, you can print more details or perform additional assertions
expect(results[0]).toHaveProperty('name', 'react');
expect(results[0]).toHaveProperty('version', '18.2.0');
});
});
}
},
);
Loading

0 comments on commit 1bf8193

Please sign in to comment.