Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Preserve Unicode escape sequences in transformation #173

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zermelo-wisen
Copy link
Collaborator

Fixes #172.

Ensures that Unicode escape sequences (e.g., "\u{1D306}") are correctly preserved during the code transformation process. This fix addresses an issue where certain escape sequences were incorrectly converted when parsing and regenerating code.

The fix gives raw: true option to the meriyah parse function in transform.ts.

 const tree = parse(code, {
   source: url.toString(),
   next: true,
   loc: true,
   module: true,
   onComment: comments,
   raw: true, // Added this line
 });

This test, which would fail without the fix, is added to transform.test.ts.

it("transforms unicode correctly", () => {
  const input = `const a = "\\u{1D306}";\n`;
  // Ensure that "\u{1D306}" is not converted to "t̆".
  expect(transform(input, new URL("file:///foo"), [new IdentityHook()])).toBe(input);
});

Ensure that Unicode escape sequences (e.g., "\u{1D306}") are correctly
preserved during the code transformation process. This fix addresses an
issue where certain escape sequences were incorrectly converted when
parsing and regenerating code, leading to unintended character
alterations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failing tests in gchq/CyberChef
1 participant