All notable changes to this project will be documented in this file.
- Fixed bug, which crashed the cluster in some environments (#113)
- Changed when the
queue
event is emitted (after the job is queued into the internal queue).
- Event
queue
added
- Support for generics via
Cluster<InType, OutType>
package-lock.json
file was not updated
- Internal helper function was accidentally set to public
Cluster.execute
function added- Examples for
Cluster.execute
added
- Fixed support for custom concurrency implementations
- Support for custom puppeteer libraries added
- Support for custom concurrency implementations added
- Updated dependencies to their latest versions
- Fixed another sameDomainDelay bug leading to high CPU usage
- Fixed
sameDomainDelay
bug (issue #11)
- Fixed rarely happening bug (issue #3), which made browser not able to restart
- Node.js support for version 6 and 7 added
- Circular structures led to crashs in case of crawling errors.
- Cluster.task function signature changed from
Cluster.task(page, url)
toCluster.task({ page, data })
.Cluster.queue
can be passed any data instead of a string or object.
- The timeout-promise for a task was not canceled when a task threw an error.
- Cluster can be used without defining a Cluster.task function by queuing only functions.
- Page errors ("Page crashed!") were not caught so far
- maxCPU and maxMemory options removed as they made no sense (better to check how much chromium your machine can handle.)