TODO

== Ingestion ==
- in ingestible review screen, add UNDO INGESTION button, with confirmation
- treat TOC buffer as implicitly start-to-finish if no explicit start or finish is given
- show text splits in full-markdown pane with thick horizontal border like the legacy HtmlFile ingestion
- allow creating a new periodical
- allow creating the first issue of a new periodical

== collection management ==
- in insertion, re-enable the submit button after a work is selected
- upon text insertion, an extraneous delete button appears

== older stuff, some outdated ==
! Fix A-B BAR!
אסף האם ניתן להוסיף לטופס הבקשה להתנדב שאלה ״היכן שמעת על הפרויקט? ״
יהיה נחמד לדעת אילו מהאמצים שלנו נושאים פרי (נניח רחל מדווחת שהיו הרבה אנשים בהרצאה הבוקר ושהתגובות היו נלהבות. מעניין אם מוביל לפעולה).
- refactor header views into single files (author_top -> toc) using https://guides.rubyonrails.org/layouts_and_rendering.html#structuring-layouts

1. implement news items input/editing for heterogeneous newsfeed

Data model flattening TODO:
- Replicate Work and Expression fields in Manifestation, but make them auto-update when they are edited in the Work/Expression entity, using after_save hooks.  They should never be editable manually.
- Once that's in place, remove the joins and lookups for those fields in manifestation displays, especially in /read and /author views.

ignore illustrators in gen_toc
(completely? Maybe only if they're not *also* the creators?)

  - work aboutnesses
  - incoming aboutnesses for work!

- code to find probably-typos:
  - digits within words
  - finals within words
  - what else?
  - non-title paragraphs ending without period, question mark, exclamation point.
- maintenance report (weekly rake task?) finding texts with such, adding them to a List, *unless* they're already whitelisted.
- in edit_markdown, identify probable typos and decorate them with large, colorful HTML. Some color for the whole word, and red for the suspected typo.
- in edit_markdown, add button to whitelist for probable typos.
- upon markdown save, re-check for probable typos, and if there are none and the work is in the Probable Typos list, remove it from the list.

browser history hack

GET page with params - no filters
  onload: push state: URL+page+sort
  onnav: replace state: URL+page+sort

POST page with no filters
  onload: push state: URL+page+sort
  onnav: replace state: URL+page+sort

POST page with filters
  onload: push state: filters
  onnav: replace state: filters
  onfilterchange: replace state: filters

onpopstate
 if type=filters
   deserialize form and submit
 else
   POST url with page and sort params

There is a re-entrancy problem in creating a PDF from works with images (wkhtmltopdf requests the images, and the GVL causes a deadlock).  Need to rewrite to *first* download the images to /tmp, rewrite the IMG src attributes, and then generates the PDF, taking care to enable local file access in wkhtmltopdf.)
  - for PDFs, also *scale* the local copy of images to no more than 20.9cm width, to solve PDF page width issue.

היות שהמקרה הנפוץ הוא של "מסע רכש" שמסתיים בהשגת כמה עשרות ספרים בבת אחת (וכן של קבלת קבוצת סריקות בבת אחת), אפשר ליצור מסך עדכון שבו בוחרים סטטוס פעם אחת (למשל "בידינו"), ואז בשורת חיפוש עם השלמה-אוטומטית מזינים התחלות של כותרים ופשוט לוחצים על האופציה הרצויה והסטטוס מתעדכן, ומיד אפשר לעבור להקלדת שם הכותר הבא.

א. הוספת שדה הערות לכותר בסדנה.
ב. פיתוח תמיכה ב"עלי כותרים" לתשאול בסדנה הביבליוגרפית.

EbyDict-Integration TODO

- implement definition view remaining buttons (cite etc.)
- cronjob for exports
- cronjob for imports (staging for now, eventually production)
+ link stats header to EbyDict entity (staging, prod) [eventually, link instead to genre query for lexicon, but only after figuring out what to do with non-ebydict lexica]
- add lexicon_metadata table
  - total_headwords
  - display mode (html-markdown; dictionary-entries)
  - intro text template

Older TODO
element.scrollIntoView({
    behavior: 'smooth',
    block: 'start' // scroll to top of target element
})
- editing TOCs, add button "add all works not already linked".
- add complementary query when date filter selected, counting excluded no-date results
- implement showing of excluded no-date results
- autocomplete for author names in filters
- mark items already in anthology in results list ?
- generalize user persistency over actual User records versus cookie-based semi-ephemeral saved settings

IDEA: now that we have picture support - maybe allow VARIANTS of existing manifestations, with (trusted) user-curated illustrated editions (taking from Commons etc.)
- emit pics in print/export
- Implement Capistrano deployment in production: https://github.com/capistrano/capistrano/blob/master/README.md
  - staging works

- cron or timed job to expire sessions
- cron or timed job to tally and delete impressions records
- report/task to find reversed paragraphs using hspell and a count of unrecognized words

==Bib Workshop==
grab DDC ids for Raanana through gared (async?)

- resolve narrow width issue in work read view - newlines become <p> tags for some reason!  Doesn't happen in edit mode, without the markdown.lines manipulation!

- add link type "original text URL" for translations (e.g. in Gutenberg or a foreign-language Wikisource)
- add additional translated_from field to expression, for cases where the original language of the work is not the language this expression was translated from. (e.g. Shlonsky's Hamlet translation from Russian.)

- BACKEND
  - add dropdown with expression details history *per author* to save work typing it for other expressions by that author.
  - add lock_version column to person, work, expression, and manifestation tables
  - implement taggings review
  - EPIC: have works automatically tagged as belonging to a book/collection (WEM!) according to (human-confirmed) heading from the ToC
  - show revisions of manifestations/ToCs
  - show diffs between revs
  - restore rev over current
  - query work by tag
  - admin review, approve/reject tags and recommendations

- !! add translations box to works view right side
- make tags links to works queries by tag (query already works, at /tag/:id)
- implement recommendation flagging (data model, review)
- implement those-who-read-this-also-read-that (i.e. https://measuringu.com/affinity-analysis/)
- implement references-to-this-work
- (?) other extras at bottom of work read
- implement chapter-only display
- implement WhatsApp Web when detected Desktop
- implement favoriting authors
- download as TXT - add CR+LF if user runs Windows
+ download as DOC - right-align (pandoc patch?)
- download as any format -- ensure title and authorship metadata properly emitted (apparently not currently the case in PDF?)

- INTEROP
  - Be FAIR - https://www.go-fair.org/fair-principles/

- HOMEPAGE
  - featured content full text view
  - carousel of featured content

- homepage bottom -- popular authors *this month*
  - need to calculate in controller
- process ToCs
  - process bad ToCs (bad_tocs.txt)

- SEARCH
  - literal search, including quoted phrases
  - remove ' and " during indexing, and from query strings (" only from *middle* of words, e.g. מפא"י, to still allow quoted phrases search)
  - weight work titles highest after authors
- SEARCH RESULTS
  - implemented sort/filter (like the browse views in manifestations)
    - how to deal with heterogeneous results, though?

- BACKEND
  - UX throughout
  - missing maintenance reports

- user prefs infrastructure for anons!
  + store fontsize pref

* always: rake notes
* fix RTL alignment in downloaded DOC files
* set fonts in HTML, DOC, PDF(?), EPUB(?), MOBI(?)
check HtmlFile id 4291 in production (Sokolov) -- currently(??) crashing on charset
- maybe switch to a print CSS rather than a print action/view?  crucial code is @media print {
  div.info, div#editStuff, div#navigation, div#debug {
    display: none;
  }
}

- chop_head didn't work on negev
- kabak's ToC didn't update
- parallel with original in render_html

Conversion system
-----------------
- detect indentation? (poetry, letter signatures, etc.)
- detect size changes
- handle ul and ol tags?
- handle titles
+ handle subtitles
  - test on greater variety of files to ensure it works
+ handle entities, and escaping
+ handle asterisk-separator-lines as horizontal rules
+ handle En-dash horizontal rules
- detect and do something intelligent about extraneous spaces at beginning of paragraphs, avoiding letting them become PRE blocks
- gather info for later cataloguing -- title, author, translator, copyright status, dedication/epigraph, year/location, ...?
- add markdown manipulations post-parsing
-+ chop title/author lines from top
-+ treat as poetry (<code><pre>)
-+ compress spaces
-- raw edit (versioned?  just backup/undo?)
- when publishing a parsed HtmlFile:
  - allow to verify/edit title and author
  - allow to insert magic sequences to break between different Works in the HtmlFile (think short poems), and post-submit show a form for titles for each before publishing separate Manifestations and Works for each
- support a configurable set of auto fixes for post-processing markdown
  - e.g. match_all nun-sofit+non-whitespace, offer to replace with vav
  - e.g. match_all ' " ' for mis-spaced quotes
  - e.g. match sofiyot in middle of words
- figure out if the NLI can give us anything useful via Z39.50 re author identities.  http://web.nli.org.il/sites/NLI/English/infochannels/librarians/Pages/Z39.50.aspx  (on the face of it, it doesn't look as though we can query the authority file itself!)
  - other useful URLs: http://www.loc.gov/z3950/agency/defns/bib1.html  http://ruby-zoom.rubyforge.org/ https://github.com/bricestacey/ruby-zoom/blob/master/sample/needle.rb

+ add way to bless markdown by human reviewer
-- blessing should, whether automatically or as an extra human-chosen step, also post the blessed markdown (with path info etc.) to the CONTENT system, so it can be rendered from it INSTEAD of the old HTML file (preserving the *.html URL)
- research and possibly integrate good editor -- or just spiff up ContentEditable?

Proof and Recommendation feature
--------------------------------
- add JS input for blog url when announcing blogged recommendation
- add auto-expiration (or hiding?) of wontfix proofs and rejected recommendations

+ figure out a stable representation for poetry with line breaks
  + PRE blocks ignore *bold* markup, so are probably infeasible
  + blockquotes (lines beginning with '>' do honor bold and other markup, but the linebreak behavior seems inconsistent -- it appears the double-space-and-line-break method does not consistently produce line breaks *inside* a blockquote (but *does* before/after an explicit <br> tag!), so probably <br> tags are necessary to ensure space lines between stanzas.  Needs further experimentation to determine how to consistently generate just the right amount of space.

== Epics ==
* Add TESTS