Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests with all Node frameworks, fix ReadableStream.cancel & remove URL ponyfill #1929

Merged
merged 25 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/@whatwg-node_fetch-1929-dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@whatwg-node/fetch": patch
---
dependencies updates:
- Updated dependency [`@whatwg-node/node-fetch@^0.7.5` ↗︎](https://www.npmjs.com/package/@whatwg-node/node-fetch/v/0.7.5) (from `^0.7.1`, in `dependencies`)
6 changes: 6 additions & 0 deletions .changeset/@whatwg-node_node-fetch-1929-dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@whatwg-node/node-fetch": patch
---
dependencies updates:
- Removed dependency [`@kamilkisiela/fast-url-parser@^1.1.4` ↗︎](https://www.npmjs.com/package/@kamilkisiela/fast-url-parser/v/1.1.4) (from `dependencies`)
- Removed dependency [`fast-querystring@^1.1.1` ↗︎](https://www.npmjs.com/package/fast-querystring/v/1.1.1) (from `dependencies`)
31 changes: 31 additions & 0 deletions .changeset/warm-plums-grin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
'@whatwg-node/node-fetch': patch
---

- Remove URL ponyfill implementation based on `fast-url-parser` and `fast-querystring`, because Node now uses Ada URL parser which is fast enough.

- Fix `ReadableStream[Symbol.asyncIterator]`

`ReadableStream` uses `Readable` so it uses `Symbol.asyncIterator` method of `Readable` but the returned iterator's `.return` method doesn't handle cancellation correctly. So we need to call `readable.destroy(optionalError)` manually to cancel the stream.

This allows `ReadableStream` to use implementations relying on `AsyncIterable.cancel` to handle cancellation like `Readable.from`

Previously the following was not handling cancellation;

```ts
const res = new ReadableStream({
start(controller) {
controller.enqueue('Hello');
controller.enqueue('World');
},
cancel(reason) {
console.log('cancelled', reason);
}
});

const readable = Readable.from(res);

readable.destroy(new Error('MY REASON'));

// Should log 'cancelled MY REASON'
```
3 changes: 2 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@
"website",
"scripts",
"e2e",
"benchmarks"
"benchmarks",
"deno-jest.ts"
],
"globals": {
"BigInt": true
Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/deployment-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
strategy:
fail-fast: false
matrix:
plan: ['aws-lambda', 'azure-function', 'cloudflare-workers', 'cloudflare-modules', 'deno']
plan: ['aws-lambda', 'azure-function', 'cloudflare-workers', 'cloudflare-modules']
# TODO: Add vercel
name: e2e / ${{ matrix.plan }}

Expand All @@ -27,12 +27,6 @@ jobs:
- name: Install Required Libraries
run: sudo apt update && sudo apt install -y libcurl4-openssl-dev libssl-dev

- name: Use Deno
if: matrix.plan == 'deno'
uses: denoland/setup-deno@v2
with:
deno-version: vx.x.x

- name: Cache Node Modules
uses: actions/cache@v4
id: node-modules-cache-deployment-e2e
Expand Down
36 changes: 26 additions & 10 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:

env:
NODE_OPTIONS: --max-old-space-size=4096
NODE_NO_WARNINGS: 1
CI: true

jobs:
Expand Down Expand Up @@ -97,15 +98,13 @@ jobs:
command: yarn test:leaks --ci

unit-bun:
name: unit / bun ${{matrix.node-version}}
name: unit / bun
runs-on: ubuntu-latest
services:
httpbin:
image: kennethreitz/httpbin
ports:
- 8888:80
strategy:
fail-fast: false
steps:
- name: Checkout Master
uses: actions/checkout@v4
Expand All @@ -115,16 +114,33 @@ jobs:
uses: the-guild-org/shared-config/setup@main
with:
nodeVersion: 22
- name: Cache Jest
uses: actions/cache@v4
with:
path: .cache/jest
key: ${{ runner.os }}-${{matrix.node-version}}-jest-${{ hashFiles('yarn.lock') }}
restore-keys: |
${{ runner.os }}-${{matrix.node-version}}-jest-
- name: Test
run: yarn test:bun --ci

unit-deno:
name: unit / deno
runs-on: ubuntu-latest
services:
httpbin:
image: kennethreitz/httpbin
ports:
- 8888:80
steps:
- name: Checkout Master
uses: actions/checkout@v4
- name: Install Required Libraries
run: sudo apt update && sudo apt install -y libcurl4-openssl-dev libssl-dev
- name: Setup env
uses: the-guild-org/shared-config/setup@main
with:
nodeVersion: 22
- name: Test
uses: nick-fields/retry@v3
with:
timeout_minutes: 10
max_attempts: 5
command: yarn test:deno

esm:
name: esm
runs-on: ubuntu-latest
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,5 @@ junit.xml
package-lock.json

eslint_report.json

deno.lock
1 change: 1 addition & 0 deletions .yarnrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ignore-engines true
2 changes: 1 addition & 1 deletion benchmarks/node-fetch/server.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { createServer } from 'http';
import { createServer } from 'node:http';
import { isScenario, scenarios } from './scenarios';

const port = 50001;
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/server/server.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Blob as BufferBlob } from 'buffer';
import { createServer, type RequestListener } from 'http';
import { Blob as BufferBlob } from 'node:buffer';
import { createServer, type RequestListener } from 'node:http';
import * as undici from 'undici';
import { createServerAdapter, Response } from '@whatwg-node/server';

Expand Down
22 changes: 22 additions & 0 deletions deno-jest.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import { fn } from 'jsr:@std/expect';

export {
describe,
it,
test,
beforeEach,
afterEach,
beforeAll,
afterAll,
} from 'jsr:@std/testing/bdd';
export { expect } from 'jsr:@std/expect';

export const jest = {
fn,
spyOn(target: any, method: string) {
Object.defineProperty(target, method, {
value: fn(target[method]),
writable: true,
});
},
};
13 changes: 13 additions & 0 deletions deno.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"imports": {
"@jest/globals": "./deno-jest.ts",
"@whatwg-node/cookie-store": "./packages/cookie-store/src/index.ts",
"@whatwg-node/disposablestack": "./packages/disposablestack/src/index.ts",
"@whatwg-node/fetch": "./packages/fetch/dist/esm-ponyfill.js",
"@whatwg-node/events": "./packages/events/src/index.ts",
"fetchache": "./packages/fetchache/src/index.ts",
"@whatwg-node/node-fetch": "./packages/node-fetch/src/index.ts",
"@whatwg-node/server": "./packages/server/src/index.ts",
"@whatwg-node/server-plugin-cookies": "./packages/server-plugin-cookies/src/index.ts"
}
}
2 changes: 1 addition & 1 deletion e2e/aws-lambda/scripts/createAwsLambdaDeployment.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from 'node:path';
import {
assertDeployedEndpoint,
DeploymentConfiguration,
Expand Down
1 change: 1 addition & 0 deletions e2e/aws-lambda/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { Buffer } from 'node:buffer';
import { APIGatewayEvent, APIGatewayProxyResult, Context } from 'aws-lambda';
import { createTestServerAdapter } from '@e2e/shared-server';
import { URL } from '@whatwg-node/fetch';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from 'node:path';
import {
assertDeployedEndpoint,
DeploymentConfiguration,
Expand Down
2 changes: 1 addition & 1 deletion e2e/cloudflare-workers/scripts/createCfDeployment.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { join } from 'path';
import { join } from 'node:path';
import {
assertDeployedEndpoint,
DeploymentConfiguration,
Expand Down
23 changes: 0 additions & 23 deletions e2e/deno/e2e.ts

This file was deleted.

Loading
Loading