[pull] main from caikit:main #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Need to make sentence-transformers part of the default caikit-nlp runtime image. Previously this was an optional import requiring a special runtime image. * Make sentence-transformers one of the dependencies * Remove the just-for-testing no-deps support that was a work-around Signed-off-by: markstur <[email protected]>
Signed-off-by: markstur <[email protected]>
…runcation Wrapped model with custom encode params. Moves truncation into encode (tokenize) processing). Adds option to use dtype=bloat16 with autocast (adding this for IPEX testing). Use env var BFLOAT16 to enable the dtype for ipex and the autocast in encode. Signed-off-by: markstur <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: Shonda-Adena-Witherspoon <[email protected]>
…task in decorator Signed-off-by: Shonda-Adena-Witherspoon <[email protected]>
…e.py Signed-off-by: Shonda-Adena-Witherspoon <[email protected]>
* add tests and verify that we get the expected truncation errors * re-tokenize to recreate the token count on trucation errors * cleanup unused stuff Signed-off-by: markstur <[email protected]>
Signed-off-by: Shonda-Adena-Witherspoon <[email protected]>
Add sentence-transformers to dependencies for real
added tokenization tasks to serve tokenization requests
Signed-off-by: waleedqk <[email protected]>
git push origin $BRANCH e remote-tracking branch 'origin/main' into ResponseOptions
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: pull[bot] The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Hi @pull[bot]. Thanks for your PR. I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: markstur <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
* config introduces values that env vars wouldn't * make those checks more robust * make it more testable Signed-off-by: markstur <[email protected]>
* For doc/defaults Signed-off-by: markstur <[email protected]>
Signed-off-by: markstur <[email protected]>
Wrapped sentence transformer
Signed-off-by: Mynhardt Burger <[email protected]>
Signed-off-by: Mynhardt Burger <[email protected]>
…in-results Include input token count in results
🥅 Allow int thresholds
* Adding missing params * Don't return unexpected tuple (with token count) unless asked * Adding check to not use our params if given an unwrapped model * Fixing some param position things Signed-off-by: Mark Sturdevant <[email protected]>
…tion behavior. * First draft could KeyError when deleting kwargs that don't exist. Tests added. * Adding a config option so the desired default behavior can be either: - Throw an error if truncation is happening implicitly, or - Nah. Just let it go. First was requested, so trunction does not happen quietly. This is common behavoir for some models. The second is more aligned with SentenceTransformers and is probably necessary for standard tests to run without errors. Signed-off-by: Mark Sturdevant <[email protected]>
Signed-off-by: Mark Sturdevant <[email protected]>
Make encode() in wrapped model compatible with super encode()
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
⚡ Tee stream instead of checking length
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Adding tgis params to caikit api
…h truncation * Attempting truncate_input_tokens=2 (or 1) was creating a strange error (or misbehaving) because it takes at least 3 tokens for [CLS] TOK [SEP] for meaningful results. * Now that truncate value generally means number of tokens not including begin/end. * On the max end the 2 special tokens will be allowed to consume 2 from the limit. * Batch embedding processing was returning odd/misordered results when combined with truncation. Added a re tokenize() call to avoid sending the overflow tokens as features to be processed. Signed-off-by: Mark Sturdevant <[email protected]>
Embeddings fix for truncation without room for begin/end and for batch truncation
* Test for 2 identical strings to return identical vectors * Also tests that batch vs single/loop are approx same * This test would have found a bug that was recently fixed Signed-off-by: Mark Sturdevant <[email protected]>
Embedding add a test that would have helped
Signed-off-by: Evaline Ju <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
Signed-off-by: Evaline Ju <[email protected]>
✨ Add tokenization task to generation modules
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )