Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bedrock-runtime client apply_guardrail function adds new line at the end of masked text #4316

Closed
1 task
motti-trail opened this issue Oct 26, 2024 · 1 comment
Labels
bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged.

Comments

@motti-trail
Copy link

Describe the bug

When using bedrock-runtime apply_guardrail on a given text, where guardrail is configured with mask action, the returned text is indeed masked where needed, but a new line is added to the given input.

TEXT = "The official address of the UK Prime Minister\'s office is:\\n\\n10 Downing Street\\nLondon\\nSW1A 2AA\\nUnited Kingdom\\n\\nThis address is commonly referred to simply as \\"10 Downing Street\\" or \\"Number 10.\\" It has been the official residence and office of the British Prime Minister since 1735. The building is located in central London, near the Houses of Parliament and Whitehall."

bedrock = boto3.client(...).   # bedrock has a configured guardrails with "ADDRESS" PII and action "mask"
guard_res = bedrock.apply_guardrail(
    guardrailIdentifier=...,
    guardrailVersion=..., 
    source="OUTPUT",
    content=[{'text':{'text':TEXT}}]
)

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

Bedrock should return the text as is, while masking the configured addresses without a newline at the end.

guard_res['outputs'][0]['text'] should be:

"The official address of the UK Prime Minister\'s office is:\n\n{ADDRESS}\n{ADDRESS}\n\nThis address is commonly referred to simply as "{ADDRESS}." It has been the official residence and office of the British Prime Minister since 1735. The building is located in central {ADDRESS}, near the Houses of Parliament and Whitehall."

Current Behavior

Actual results (guard_res['outputs'][0]['text']):

"The official address of the UK Prime Minister\'s office is:\n\n{ADDRESS}\n{ADDRESS}\n\nThis address is commonly referred to simply as "{ADDRESS}." It has been the official residence and office of the British Prime Minister since 1735. The building is located in central {ADDRESS}, near the Houses of Parliament and Whitehall.\n"

As you can see when scrolling to the end of the resulting text, a newline character \n was added to the end of the text.

Reproduction Steps

TEXT = "The official address of the UK Prime Minister\'s office is:\\n\\n10 Downing Street\\nLondon\\nSW1A 2AA\\nUnited Kingdom\\n\\nThis address is commonly referred to simply as \\"10 Downing Street\\" or \\"Number 10.\\" It has been the official residence and office of the British Prime Minister since 1735. The building is located in central London, near the Houses of Parliament and Whitehall."

bedrock = boto3.client(...).   # bedrock has a configured guardrails with "ADDRESS" PII and action "mask"
guard_res = bedrock.apply_guardrail(
    guardrailIdentifier=...,  # add guardrail id
    guardrailVersion=...,  # add guardrail version or 'DRAFT'
    source="OUTPUT",
    content=[{'text':{'text':TEXT}}]
)

print(guard_res['outputs'][0]['text'].endswith('\n') is True)
print(guard_res['outputs'][0]['text'])

Possible Solution

I'm unsure of the root cause of this behavior, but I see that the client already got the new line char as a response to the API call.

Additional Information/Context

I also checked the usage of configuredGuardrail in the call for the converse function. The behavior didn't reproduce, and the guarded text returned as expected.

SDK version used

1.34.162

Environment details (OS name and version, etc.)

MacOS 14.7

@motti-trail motti-trail added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Oct 26, 2024
@motti-trail motti-trail closed this as not planned Won't fix, can't repro, duplicate, stale Oct 26, 2024
Copy link

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant