Skip to content

Commit

Permalink
Merge pull request #12 from alpsla/analysis_pipeline
Browse files Browse the repository at this point in the history
completed Analysis Pipeline
  • Loading branch information
alpsla authored Dec 30, 2024
2 parents 58577e6 + 4aace9d commit df59d6d
Show file tree
Hide file tree
Showing 44 changed files with 5,112 additions and 2,093 deletions.
126 changes: 116 additions & 10 deletions Docs/implementation-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,18 +203,19 @@
- [x] Unsupported file handling
- [x] Comprehensive test coverage
- [ ] LLM Integration
- [x] LLM Integration
- [x] Base prompt templates
- [x] Language-specific analysis prompts
- [ ] Repository analysis templates
- [ ] Response parsing and formatting
- [ ] Error handling and fallbacks
- [ ] Analysis Pipeline
- [ ] Queue management for analysis jobs
- [ ] Progress tracking and status updates
- [ ] Result storage and caching
- [ ] Retry mechanisms for failed analyses
- [x]Repository analysis templates
- [x] Response parsing and formatting
- [x] Error handling and fallbacks
- [x] Analysis Pipeline
- [x] Queue management for analysis jobs
- [x] Progress tracking and status updates
- [x] Result storage and caching
- [x] Retry mechanisms for failed analyses
- [ ] PR Input and Validation
- [ ] PR URL parsing and validation
- [ ] Repository access verification
Expand Down Expand Up @@ -530,3 +531,108 @@ SLACK_WEBHOOK= # For deployment notifications
### PR Analysis Setup [In Progress]
- [x] Language Analysis System
- [x] File extension detection and mapping
- [x] Framework and build tool detection
- [x] Accurate percentage calculations with deduplication
- [x] Unsupported file handling
- [x] Comprehensive test coverage
- [x] LLM Integration
- [x] Basic prompt templates
- [x] Language-specific analysis prompts
- [x] Repository analysis templates
- [x] Response parsing and validation
- [x] Error handling and fallbacks
- [ ] Analysis Pipeline [Current Focus]
- [x] Queue management system
- [x] GitHub API integration with rate limiting
- [x] Repository data fetching
- [x] Contributor statistics
- [x] Commit frequency analysis
- [x] Issue and PR metrics
- [x] Rate limiting and error handling
- [x] Smart rate limit tracking
- [x] Automatic retries
- [x] Exponential backoff
- [ ] Analysis processing implementation
- [x] Repository metadata collection
- [ ] Code analysis
- [ ] LLM prompt generation
- [ ] Response processing
- [ ] Progress tracking and status updates
- [ ] Result storage and caching
- [ ] Retry mechanisms for failed analyses
- [ ] PR Input and Validation
- [ ] PR URL parsing and validation
- [ ] Repository access verification
- [ ] Error handling for invalid/inaccessible PRs
- [ ] Basic PR input form UI
### GitHub Data Fetching [In Progress]
- [x] Core Repository Data
- [x] Repository metrics (stars, forks, issues)
- [x] Repository metadata
- [x] Language statistics
- [x] Contributors statistics
- [x] Issue resolution times
- [x] PR merge times
- [ ] Advanced Repository Data
- [ ] Branch protection rules
- [x] Commit history patterns
- [x] Code frequency analysis
- [ ] PR-Specific Data
- [ ] Diff content analysis
- [ ] Review history
- [ ] CI/CD results
- [ ] Test coverage changes
- [ ] Related issues/PRs
### Database Implementation [Planned]
- [ ] Schema Design
- [ ] Repository analysis tables
- [ ] PR analysis tables
- [ ] User metrics tables
- [ ] Team metrics tables
- [ ] Data Management
- [ ] Analysis results storage
- [ ] Historical data tracking
- [ ] Cache management
- [ ] Data cleanup policies
### Progress Tracking [Planned]
- [ ] User Metrics
- [ ] Skill development tracking
- [ ] Contribution analysis
- [ ] Code quality trends
- [ ] Team Metrics
- [ ] Velocity tracking
- [ ] Collaboration patterns
- [ ] Quality improvement trends
### Future Enhancements
- [ ] Automated fix suggestions
- [ ] Documentation generation
- [ ] Team performance dashboards
- [ ] Skill-based recommendations
### Technical Debt & Maintenance
- [x] Rate limiting improvements
- [x] Smart rate tracking
- [x] Automatic retries
- [x] Request queueing
- [ ] Performance optimization
- [ ] Error handling refinement
- [ ] Test coverage expansion
- [ ] Documentation updates
8 changes: 8 additions & 0 deletions apps/api/src/tests/sample.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
var globals_1 = require("@jest/globals");
(0, globals_1.describe)('Sample Test', function () {
(0, globals_1.it)('should pass', function () {
(0, globals_1.expect)(true).toBe(true);
});
});
7 changes: 7 additions & 0 deletions apps/web/__mocks__/jose.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// Mock jose module
module.exports = {
decodeJwt: jest.fn(),
SignJWT: jest.fn(),
jwtVerify: jest.fn(),
createRemoteJWKSet: jest.fn()
};
11 changes: 11 additions & 0 deletions apps/web/__mocks__/preact-render-to-string.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
const renderToString = jest.fn().mockImplementation((component) => 'mock-rendered-string');
const render = jest.fn().mockImplementation((component) => 'mock-rendered-string');
const shallowRender = jest.fn().mockImplementation((component) => 'mock-rendered-string');

module.exports = {
renderToString,
render,
shallowRender,
default: render,
__esModule: true
};
34 changes: 34 additions & 0 deletions apps/web/__mocks__/preact.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
const h = jest.fn((type, props, ...children) => ({
type,
props: { ...props, children: children.length ? children : undefined }
}));

const Fragment = Symbol.for('preact.Fragment');
const Component = class {};
const createContext = jest.fn(() => ({
Provider: jest.fn(),
Consumer: jest.fn()
}));
const createRef = jest.fn(() => ({ current: null }));
const render = jest.fn();
const hydrate = jest.fn();
const isValidElement = jest.fn(() => true);
const cloneElement = jest.fn((element) => ({ ...element }));
const toChildArray = jest.fn((children) => [].concat(children).filter(Boolean));
const options = {};

module.exports = {
h,
Fragment,
Component,
createContext,
createElement: h,
createRef,
hydrate,
isValidElement,
options,
render,
toChildArray,
cloneElement,
__esModule: true
};
18 changes: 18 additions & 0 deletions apps/web/babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
module.exports = {
presets: [
['@babel/preset-env', {
targets: { node: 'current' },
modules: 'commonjs'
}],
'@babel/preset-typescript',
],
plugins: [
['@babel/plugin-transform-runtime', {
corejs: { version: 3, proposals: true },
helpers: true,
regenerator: true,
useESModules: false,
version: '^7.26.0'
}],
],
};
42 changes: 37 additions & 5 deletions apps/web/jest.config.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
const nextJest = require('next/jest');

const createJestConfig = nextJest({
// Provide the path to your Next.js app to load next.config.js and .env files in your test environment
dir: './',
});

// Add any custom config to be passed to Jest
const customJestConfig = {
setupFilesAfterEnv: [
'<rootDir>/tests/setup/test-config.ts',
Expand All @@ -17,7 +15,6 @@ const customJestConfig = {
'<rootDir>/src/**/__tests__/**/*.test.{js,jsx,ts,tsx}',
'<rootDir>/tests/**/*.test.{js,jsx,ts,tsx}'
],
// Add more setup options before each test is run
clearMocks: true,
collectCoverage: true,
collectCoverageFrom: [
Expand All @@ -43,8 +40,43 @@ const customJestConfig = {
'/supabase/',
'/tests/integration/supabase',
'/src/lib/supabase'
]
],
transformIgnorePatterns: [
'/node_modules/(?!(next|@next|@babel|@swc|uuid|nanoid|@auth|@vercel|@edge-runtime|cookie|@microsoft|@azure|@aws-sdk|@google-cloud|@octokit|@actions|@primer|preact.*|next-auth)).*/'
],
moduleNameMapper: {
'^jose$': '<rootDir>/__mocks__/jose.js',
'^preact-render-to-string$': '<rootDir>/__mocks__/preact-render-to-string.js',
'^preact$': '<rootDir>/__mocks__/preact.js',
'^preact/(.*)$': '<rootDir>/__mocks__/preact.js'
},
transform: {
'^.+\\.(js|jsx|ts|tsx|mjs)$': ['@swc/jest', {
jsc: {
parser: {
syntax: 'typescript',
tsx: true,
decorators: true,
},
transform: {
react: {
runtime: 'automatic'
}
}
}
}]
},
extensionsToTreatAsEsm: ['.ts', '.tsx'],
maxWorkers: 1,
testTimeout: 60000,
forceExit: true,
detectOpenHandles: true,
globals: {
'ts-jest': {
isolatedModules: true,
useESM: true
}
}
};

// createJestConfig is exported this way to ensure that next/jest can load the Next.js config which is async
module.exports = createJestConfig(customJestConfig);
8 changes: 8 additions & 0 deletions apps/web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
},
"dependencies": {
"@next-auth/prisma-adapter": "^1.0.7",
"@octokit/plugin-rest-endpoint-methods": "^13.2.6",
"@octokit/rest": "^20.0.2",
"@prisma/client": "^5.7.1",
"@radix-ui/react-toast": "^1.2.4",
Expand All @@ -31,8 +32,15 @@
"zod": "^3.22.4"
},
"devDependencies": {
"@babel/core": "^7.26.0",
"@babel/plugin-transform-modules-commonjs": "^7.26.3",
"@babel/plugin-transform-runtime": "^7.25.9",
"@babel/preset-env": "^7.26.0",
"@babel/preset-typescript": "^7.26.0",
"@babel/runtime-corejs3": "^7.26.0",
"@octokit/types": "^13.6.2",
"@supabase/postgrest-js": "^1.17.7",
"@swc/jest": "^0.2.37",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^14.3.1",
"@testing-library/user-event": "^14.5.2",
Expand Down
24 changes: 19 additions & 5 deletions apps/web/prisma/schema.prisma
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema

generator client {
provider = "prisma-client-js"
}
Expand Down Expand Up @@ -57,6 +54,23 @@ model VerificationToken {
}

model KeyValueStore {
key String @id
value String @db.Text
id String @id @default(cuid())
key String @unique
value String @db.Text
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
}

model analysis {
id String @id @default(cuid())
repositoryId String
name String
fullName String
description String
timestamp DateTime @default(now())
metrics Json
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([repositoryId])
}
4 changes: 2 additions & 2 deletions apps/web/src/app/api/__tests__/analyze.test.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import { POST } from '../analyze/route';
import { getServerSession } from 'next-auth';
import * as nextAuth from 'next-auth';
import { AnalysisPipeline } from '../../../lib/github/services/analysis-pipeline';
import type { AnalyzeRequest } from '../analyze/route';

// Mock next-auth
jest.mock('next-auth');
const mockGetServerSession = getServerSession as jest.MockedFunction<typeof getServerSession>;
const mockGetServerSession = jest.spyOn(nextAuth, 'getServerSession') as jest.MockedFunction<typeof nextAuth.getServerSession>;

// Mock services
jest.mock('../../../lib/github/services/analysis-pipeline');
Expand Down
Loading

0 comments on commit df59d6d

Please sign in to comment.