Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce improvements to AI process #651

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Conversation

AgentIsai
Copy link
Contributor

@AgentIsai AgentIsai commented Jan 21, 2025

This change introduces:

  • Forward to AI the number of requests a wiki requester has made.
  • Add new "confidence score"
  • Reduce number of parameters forwarded to the job, instead getting the necessary information inside the job itself
  • Return error messages
  • Implement forking question parameter forwarding

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced AI-powered wiki request evaluation with more detailed feedback.
    • Added confidence percentage to AI decision-making process.
    • Introduced a method to retrieve the requester's username.
  • Improvements

    • Improved error handling for wiki request evaluations.
    • More informative error messages for users, including specific reasons for failures.
    • Streamlined data management for wiki request processing.
  • Bug Fixes

    • Resolved issues with AI request evaluation logging and reporting.

The update provides more transparent and user-friendly AI-assisted wiki request processing, with clearer communication about evaluation outcomes and potential issues.

Copy link

coderabbitai bot commented Jan 21, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The pull request introduces a significant refactoring of the wiki request management system, focusing on data handling and error reporting. The changes consolidate multiple individual parameters into a single extraData array across different classes, simplifying data management. Enhancements to internationalization (i18n) files improve error messaging and AI decision feedback, adding more detailed information about request evaluations. A new method for retrieving the requester's username has also been added to improve system functionality.

Changes

File Change Summary
i18n/en.json - Added new error message keys for AI evaluation failures
- Updated AI decision dry run message to include confidence percentage
i18n/qqq.json - Added new entries for error handling related to AI interactions
includes/Jobs/RequestWikiRemoteAIJob.php - Replaced multiple properties with extraData array
- Updated method signatures to use new data structure
- Enhanced error handling and logging
includes/Services/WikiRequestManager.php - Refactored evaluateWithOpenAI to use extraData
- Added getRequesterUsername method

Sequence Diagram

sequenceDiagram
    participant User
    participant WikiRequestManager
    participant RequestWikiRemoteAIJob
    participant OpenAI

    User->>WikiRequestManager: Submit Wiki Request
    WikiRequestManager->>RequestWikiRemoteAIJob: Create Job with extraData
    RequestWikiRemoteAIJob->>OpenAI: Query AI Evaluation
    OpenAI-->>RequestWikiRemoteAIJob: Return Decision
    RequestWikiRemoteAIJob->>WikiRequestManager: Process AI Decision
    WikiRequestManager-->>User: Provide Feedback
Loading

Poem

🐰 A Rabbit's Ode to Code Refactor 🔧

With arrays so neat and trim,
Our data flows on a whim
Confidence tracked, errors clear
Code evolution brings us cheer!
Hop, hop, hooray for clean design! 🚀


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor Author

@AgentIsai AgentIsai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai review

@AgentIsai
Copy link
Contributor Author

@coderabbitai review

Check commit and GitHub actions for more details
Copy link

coderabbitai bot commented Jan 21, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ffe2cb9 and fd92022.

📒 Files selected for processing (3)
  • i18n/en.json (1 hunks)
  • includes/Jobs/RequestWikiRemoteAIJob.php (9 hunks)
  • includes/Services/WikiRequestManager.php (4 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
includes/Services/WikiRequestManager.php (1)

Pattern **/*.php: Follow MediaWiki PHP Coding Standards:

Indentation:

  • Use tabs for indentation, not spaces.

Spaces:

  • Use spaces on either side of binary operators.
    Example: $a = $b + $c; (Correct) $a=$b+$c; (Incorrect)
  • Add spaces inside parentheses, except where they are empty.
    Example: $a = getFoo( $b ); (Correct) $a = getFoo($b); (Incorrect)
  • Do not add a space after a function name.
    Example: $c = getBar(); (Correct) $c = getBar (); (Incorrect)

Function Return Type Hint:

  • Add a space after the colon in the function return type hint.
    Example: function square( int $x ): int { return $x * $x; }

Arrays:

  • Use spaces in brackets when declaring an array, except where the array is empty.
    Example: $a = [ 'foo', 'bar' ]; (Correct) $a = ['foo', 'bar']; (Incorrect)
  • Do not put spaces in brackets when accessing array elements.
    Example: $c = $a[0]; (Correct) $c = $a[ 0 ]; (Incorrect)

Control Structures:

  • Add a space after control structure keywords.
    Example: if ( isFoo() ) { ... } (Correct) if(isFoo()){ ... } (Incorrect)

Type Casting:

  • Do not use a space within or after the cast operator.
    Example: (int)$foo; (Correct) (int) $bar; (Incorrect)

Comments:

  • In comments, there should be one space between the # or // and the comment text.
    Example: // This is a comment. (Correct) //This is a comment. (Incorrect)

Ternary Operator:

  • Keep ternary operator expressions short and obvious.
  • For multi-line ternary expressions, place the '?' and ':' at the beginning of the new lines.

File Formatting:
- Avoid trailing whitespace on lines.

includes/Jobs/RequestWikiRemoteAIJob.php (1)

Pattern **/*.php: Follow MediaWiki PHP Coding Standards:

Indentation:

  • Use tabs for indentation, not spaces.

Spaces:

  • Use spaces on either side of binary operators.
    Example: $a = $b + $c; (Correct) $a=$b+$c; (Incorrect)
  • Add spaces inside parentheses, except where they are empty.
    Example: $a = getFoo( $b ); (Correct) $a = getFoo($b); (Incorrect)
  • Do not add a space after a function name.
    Example: $c = getBar(); (Correct) $c = getBar (); (Incorrect)

Function Return Type Hint:

  • Add a space after the colon in the function return type hint.
    Example: function square( int $x ): int { return $x * $x; }

Arrays:

  • Use spaces in brackets when declaring an array, except where the array is empty.
    Example: $a = [ 'foo', 'bar' ]; (Correct) $a = ['foo', 'bar']; (Incorrect)
  • Do not put spaces in brackets when accessing array elements.
    Example: $c = $a[0]; (Correct) $c = $a[ 0 ]; (Incorrect)

Control Structures:

  • Add a space after control structure keywords.
    Example: if ( isFoo() ) { ... } (Correct) if(isFoo()){ ... } (Incorrect)

Type Casting:

  • Do not use a space within or after the cast operator.
    Example: (int)$foo; (Correct) (int) $bar; (Incorrect)

Comments:

  • In comments, there should be one space between the # or // and the comment text.
    Example: // This is a comment. (Correct) //This is a comment. (Incorrect)

Ternary Operator:

  • Keep ternary operator expressions short and obvious.
  • For multi-line ternary expressions, place the '?' and ':' at the beginning of the new lines.

File Formatting:
- Avoid trailing whitespace on lines.

🔇 Additional comments (1)
includes/Services/WikiRequestManager.php (1)

993-993: 🛠️ Refactor suggestion

Remove trailing comma in function parameter list

Trailing commas in function parameter lists are only supported in PHP 7.3 and above. Remove the trailing comma to ensure compatibility with earlier PHP versions.

Apply this diff to remove the trailing comma:

private function evaluateWithOpenAI(
	array $extraData,
	string $subdomain,
-	string $username,
+	string $username
): void {

Likely invalid or redundant comment.

includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
includes/Services/WikiRequestManager.php Outdated Show resolved Hide resolved
includes/Services/WikiRequestManager.php Outdated Show resolved Hide resolved
i18n/en.json Outdated Show resolved Hide resolved
AgentIsai and others added 6 commits January 21, 2025 13:45
Check commit and GitHub actions for more details
Check commit and GitHub actions for more details
@AgentIsai
Copy link
Contributor Author

@coderabbitai review

Copy link

coderabbitai bot commented Jan 22, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
includes/Jobs/RequestWikiRemoteAIJob.php (1)

171-171: Add validation for confidence scores.

While the confidence score implementation is good, it would be safer to validate the confidence value to ensure it's within expected bounds (0-100).

Apply this diff to add validation:

+	private function validateConfidence( int $confidence ): int {
+		return max(0, min(100, $confidence));
+	}

	private function handleDryRun( string $outcome, string $comment, int $confidence ): bool {
+		$confidence = $this->validateConfidence($confidence);
		$outcomeMessage = $this->context->msg( 'requestwikiqueue-' . $outcome )->text();

	private function handleLiveRun( string $outcome, string $comment, int $confidence ): bool {
+		$confidence = $this->validateConfidence($confidence);
		$systemUser = User::newSystemUser( 'CreateWiki AI' );

Also applies to: 174-174, 177-177, 211-211, 227-228

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fd92022 and 860d866.

📒 Files selected for processing (4)
  • i18n/en.json (1 hunks)
  • i18n/qqq.json (1 hunks)
  • includes/Jobs/RequestWikiRemoteAIJob.php (8 hunks)
  • includes/Services/WikiRequestManager.php (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • i18n/en.json
🧰 Additional context used
📓 Path-based instructions (2)
includes/Services/WikiRequestManager.php (1)

Pattern **/*.php: Follow MediaWiki PHP Coding Standards:

Indentation:

  • Use tabs for indentation, not spaces.

Spaces:

  • Use spaces on either side of binary operators.
    Example: $a = $b + $c; (Correct) $a=$b+$c; (Incorrect)
  • Add spaces inside parentheses, except where they are empty.
    Example: $a = getFoo( $b ); (Correct) $a = getFoo($b); (Incorrect)
  • Do not add a space after a function name.
    Example: $c = getBar(); (Correct) $c = getBar (); (Incorrect)

Function Return Type Hint:

  • Add a space after the colon in the function return type hint.
    Example: function square( int $x ): int { return $x * $x; }

Arrays:

  • Use spaces in brackets when declaring an array, except where the array is empty.
    Example: $a = [ 'foo', 'bar' ]; (Correct) $a = ['foo', 'bar']; (Incorrect)
  • Do not put spaces in brackets when accessing array elements.
    Example: $c = $a[0]; (Correct) $c = $a[ 0 ]; (Incorrect)

Control Structures:

  • Add a space after control structure keywords.
    Example: if ( isFoo() ) { ... } (Correct) if(isFoo()){ ... } (Incorrect)

Type Casting:

  • Do not use a space within or after the cast operator.
    Example: (int)$foo; (Correct) (int) $bar; (Incorrect)

Comments:

  • In comments, there should be one space between the # or // and the comment text.
    Example: // This is a comment. (Correct) //This is a comment. (Incorrect)

Ternary Operator:

  • Keep ternary operator expressions short and obvious.
  • For multi-line ternary expressions, place the '?' and ':' at the beginning of the new lines.

File Formatting:
- Avoid trailing whitespace on lines.

includes/Jobs/RequestWikiRemoteAIJob.php (1)

Pattern **/*.php: Follow MediaWiki PHP Coding Standards:

Indentation:

  • Use tabs for indentation, not spaces.

Spaces:

  • Use spaces on either side of binary operators.
    Example: $a = $b + $c; (Correct) $a=$b+$c; (Incorrect)
  • Add spaces inside parentheses, except where they are empty.
    Example: $a = getFoo( $b ); (Correct) $a = getFoo($b); (Incorrect)
  • Do not add a space after a function name.
    Example: $c = getBar(); (Correct) $c = getBar (); (Incorrect)

Function Return Type Hint:

  • Add a space after the colon in the function return type hint.
    Example: function square( int $x ): int { return $x * $x; }

Arrays:

  • Use spaces in brackets when declaring an array, except where the array is empty.
    Example: $a = [ 'foo', 'bar' ]; (Correct) $a = ['foo', 'bar']; (Incorrect)
  • Do not put spaces in brackets when accessing array elements.
    Example: $c = $a[0]; (Correct) $c = $a[ 0 ]; (Incorrect)

Control Structures:

  • Add a space after control structure keywords.
    Example: if ( isFoo() ) { ... } (Correct) if(isFoo()){ ... } (Incorrect)

Type Casting:

  • Do not use a space within or after the cast operator.
    Example: (int)$foo; (Correct) (int) $bar; (Incorrect)

Comments:

  • In comments, there should be one space between the # or // and the comment text.
    Example: // This is a comment. (Correct) //This is a comment. (Incorrect)

Ternary Operator:

  • Keep ternary operator expressions short and obvious.
  • For multi-line ternary expressions, place the '?' and ':' at the beginning of the new lines.

File Formatting:
- Avoid trailing whitespace on lines.

🔇 Additional comments (6)
includes/Jobs/RequestWikiRemoteAIJob.php (2)

40-40: LGTM! Good consolidation of properties.

The addition of extraData array property helps consolidate multiple individual properties, improving code maintainability.


115-126: Well-structured error handling implementation!

The error handling has been improved with:

  • User-friendly error messages using i18n
  • Proper logging of errors
  • History tracking for debugging
  • Clear separation between public and internal error messages

Also applies to: 130-153

i18n/qqq.json (1)

125-127: LGTM! Clear and well-documented i18n messages.

The new error message entries are well-documented, providing clear context for translators about when each message is used.

includes/Services/WikiRequestManager.php (3)

677-679: LGTM! Useful convenience method.

The getRequesterUsername method provides a convenient way to get the username string directly, reducing the need for chained method calls.


985-994: LGTM! Good simplification of the API.

The evaluateWithOpenAI method has been simplified to only pass the request ID, which is a cleaner approach as all other data can be retrieved from the database using this ID.


370-374: LGTM! Correct user comparison.

The method now properly compares the username using getName() instead of comparing the UserIdentity object directly to a string.

includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
i18n/en.json Show resolved Hide resolved
includes/Jobs/RequestWikiRemoteAIJob.php Outdated Show resolved Hide resolved
"Category: '$category'. Contains content that is not safe for work? '$isNsfw'. " .
$nsfwReasonText . "Wiki request reason: " .
trim( str_replace( [ "\r\n", "\r" ], "\n", $reason ) );
$forkText = !empty( $extraData['sourceurl'] )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we bringing html into this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra security against inputs in RequestWiki. Do you think this is not necessary?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use str_replace( '\'', '\\\'', $text ) (and escape backslashes, newlines, potentially control characters, etc)

Though, even escaping is not enough (for example, clearly delinated user input can be interpreted as instructions, and this still can apply to the chat API)

// Extract response details with default fallbacks
$confidence = $apiResponse['recommendation']['confidence'] ?? '0';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be casted to an int, otherwise the functions that take this in will die if they're type hinted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants