Skip to content
New issue

Phase 2

Past due by over 10 years 33% complete
  • Have the server watch the crawlers directory for changes. All crawlers in the crawlers directory should be available to schedule via POST /job and view via GET /crawlers.
  • Add support for more complex schedules, delays, and retry rules.
  • Begin to add more transports (logger, mongodb, disk) that can be saved to.
  • Support PDF parsing
  • Have the server watch the crawlers directory for changes. All crawlers in the crawlers directory should be available to schedule via POST /job and view via GET /crawlers.
  • Add support for more complex schedules, delays, and retry rules.
  • Begin to add more transports (logger, mongodb, disk) that can be saved to.
  • Support PDF parsing
Loading