Server Side Token Counting #18
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![sweep-ai[bot]](https://avatars.githubusercontent.com/in/307814?s=60&v=4){kind=small}
Sandbox Executions |
Apply Sweep Rules to your PR?
|
4 tasks
❌ Deploy Preview for endless-chat failed.
|
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
❌ Deploy Preview for chat-for-free failed.
|
There was a problem hiding this comment.
- 把 todo 去掉
- 返回 userMessagesTokenCount 和 assistantMessagesTokenCount 有什么用？
There was a problem hiding this comment.
返回 userMessagesTokenCount 和 assistantMessagesTokenCount 有什么用？
There was a problem hiding this comment.
❌ Failed to make changes
Please join our Discord to report this issue.
There was a problem hiding this comment.
去掉 userMessagesTokenCount 和 assistantMessagesTokenCount
|
还需要在 /api/generate.ts 中加上和 components/Generator.tsx 中一样的裁剪 messages 的逻辑 |
|
❌ Failed to make changes Please join our Discord to report this issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Feedback (click)
Description
This pull request includes changes to the
src/utils/tiktoken.tsfile. It modifies thecountTokensSingleMessagefunction to calculate the token count for a single chat message, accounting for metadata tokens. It also updates thecountTokensfunction to handle different scenarios and provide more detailed token counts.Additionally, the pull request adds a TODO comment to remind developers to add or update tests for the new cases introduced in the code.
Summary
countTokensSingleMessagefunction to calculate the token count for a single chat message, accounting for metadata tokens.countTokensfunction to handle different scenarios and provide more detailed token counts.Fixes #17.
🎉 Latest improvements to Sweep:
ropelibrary to refactor Python! Check out Large Language Models are Bad at Refactoring Code. To have Sweep refactor your code, trysweep: Refactor <your_file>.py!💡 To get Sweep to edit this pull request, you can: