Merge branch 'develop'

texmacs · Dec 28, 2022 · dce23ec · dce23ec
2 parents 9ef3792 + 0552f9b
commit dce23ec
Show file tree

Hide file tree

Showing 14 changed files with 340 additions and 109 deletions.
diff --git a/README.md b/README.md
@@ -10,9 +10,8 @@ possibility of exporting them as html into the markdown file.
 The plugin has been developed for its use in two specific websites, 
 [Paperwhy](https://paperwhy.8027.org/) and appliedAI's 
 [TransferLab](https://transferlab.appliedai.de/), and can use multiple 
-extensions specific to the static website generator 
-[Hugo](https://gohugo.io/). There might still be some code very specific to 
-those sites, YMMV.
+extensions specific to the static website generator [Hugo](https://gohugo.io/). 
+There might still be some code very specific to those sites, YMMV.
 
 ## Setup
 
@@ -40,14 +39,34 @@ preferences:
   output. Set it to an empty value to deactivate wrapping.
 * `Numbered sections?` Whether to prepend section numbers to numbered section 
   titles.
-* `Export on save?` Whether to automatically export the document 
-  **overwriting** the last exported markdown file, every time the TeXmacs 
-  buffer is saved. The path to the exported file is stored in the document's 
-  *auxiliary data* either as a relative or an absolute path with respect to 
-  the document. Relative can be useful e.g. if the document is shared.
+* `Export on save?` Whether to automatically export the 
+  document **overwriting** the last exported markdown file, every time the 
+  TeXmacs buffer is saved. The path to the exported file is stored in the 
+  document's *auxiliary data* either as a relative or an absolute path with 
+  respect to the document. Relative can be useful e.g. if the document is 
+  shared.
 
 # Features
 
+## Additional TeXmacs macros
+
+Use the style package `markdown.ts` with `Document -> Style -> Add 
+package -> Markdown -> markdown` to enable:
+
+* Support for alternate image formats between TeXmacs and markdown via 
+  `<md-alt-image|tm-image|md-image>`. This is useful e.g. to provide 
+  SVG and EPS/PDF versions of images for print and web respectively.
+* Macros for labels in equation arrays. Using positioning tricks with 
+  `<htab>` results in LaTeX code that MathJax does not support, namely 
+  `\hfill`. For this reason, we provide two macros 
+  `<eqnarray-lab|lab>` and `<eqnarray-lab\*|lab>` which allow 
+  positioning of equation labels to the right in equation arrays. The first one 
+  also adds a `<label|eq:lab>` to be used in the document as 
+  `<reference|eq:lab>`, note the prefix `eq:`.
+
+  Symbols (e.g. $\Delta$ or $\star$) are not supported for these labels as of 
+  v0.6.1 of the plugin.
+
 ## Extensions to vanilla markdown
 
 Besides standard Markdown, this plugin supports the following:
@@ -63,38 +82,39 @@ Besides standard Markdown, this plugin supports the following:
 
 ## Hugo support
 
-In addition to the above, almost everything that Hugo supports can be 
-converted from TeXmacs, including setting frontmatter values and extensions 
-like footnotes and ~~striked through text~~.
-
-Setting values for the frontmatter is suported via a dedicated macro defined 
-in the style package `hugo.ts`. To use it go to `Document -> Style -> Add 
-package -> Markdown -> hugo` or use the icon in the focus bar.
-
-Now you can use the macro `<hugo-front>` to input any number of `key|value` 
-pairs as arguments, one argument each. That is: insert the tag `<hugo-front>`, 
-then use structured insert right to add two arguments and use the first for 
-the key and the second for the value. Repeat as needed. It is possible to use 
-strings, booleans (`true`, `false`) or dates (insert with `<date>`). 
-Additionally, the macro `<pdf-name>` inserts the name of the file with `.pdf` 
-appended. To enter a list, input `<tuple>` as the value and use structured 
-insert right to add items. To input a dictionary, use `<dict>` and again use 
-structured insert to create tuples of `key|value`.
+In addition to the above, almost everything that Hugo supports can be converted 
+from TeXmacs, including setting frontmatter values and extensions like 
+footnotes and ~~striked through text~~.
+
+Use the style package `hugo.ts` with `Document -> Style -> Add 
+package -> Markdown -> hugo` to enable:
+
+* All of the markdown extensions above
+* Setting of values for the frontmatter. With `<hugo-front>` one can 
+  input any number of `key|value` pairs as arguments, one argument each. That 
+  is: insert the tag `<hugo-front>`, then use structured insert right 
+  to add two arguments and use the first for the key and the second for the 
+  value. Repeat as needed. It is possible to use strings, booleans (`true`, 
+  `false`) or dates (insert with `<date>`). Additionally, the macro 
+  `<pdf-name>` inserts the name of the file with `.pdf` appended. To 
+  enter a list, input `<tuple>` as the value and use structured insert 
+  right to add items. To input a dictionary, use `<dict>` and again 
+  use structured insert to create tuples of `key|value`.
 
 ### Supported shortcodes
 
 All custom shortcodes are in `extensions/hugo/`.
 
-* Figures are converted to `{{< figure … >}}`.
-* For arbitrary shortcodes, use `<hugo-short>`, e.g. `<hugo-short|toc>` for 
-  `{{< toc >}}`.
-* Citations using `<cite>` and `<cite-detail>` (of any arity) are 
-  automatically detected and converted to `{{< cite ref >}}`, and all 
-  of them are gathered in the frontmatter as well, for indization by Hugo's 
+* Figures are converted to `{{< figure  >}}`.
+* For arbitrary shortcodes, use `<hugo-short>`, e.g. 
+  `<hugo-short|toc>` for `{{< toc >}}`.
+* Citations using `<cite>` and `<cite-detail>` (of any arity) 
+  are automatically detected and converted to `{{< cite ref >}}`, and 
+  all of them are gathered in the frontmatter as well, for indization by Hugo's 
   taxonomy system. The rendering of bibliography is done by `{{< 
   references >}}`.
-* Marginal notes and figures are converted to `{{< sidenote … >}}` 
-  and `{{< sidefigure … >}}`.
+* Marginal notes and figures are converted to `{{< sidenote  >}}` and 
+  `{{< sidefigure  >}}`.
 
 # Known issues
 
@@ -106,11 +126,13 @@ GitHub](https://github.com/texmacs/markdown/issues/).
   “handled”).
 * Error reporting is rather lacking. Run TeXmacs in a console to see stack 
   traces and such in case you are running into problems.
-* EPS and PDF images are not supported by browsers. As a workaround allowing 
-  to have both, use `<md-alt-image>`. The first argument is for TeXmacs and 
-  the second one for the markdown export.
+* EPS and PDF images are not supported by browsers. As a workaround allowing to 
+  have both, use `<md-alt-image>` in package `markdown.ts` as 
+  explained above. The first argument is for TeXmacs and the second one for the 
+  markdown export.
 * The converter supports only one style. For instance all environments have 
-  emphasized bodies and bold names, `<section>` is exported as `h1`, etc.
+  emphasized bodies and bold names, `<section>` is exported as `h1`, 
+  etc.
 
 # License
 

diff --git a/doc/markdown.en.tm b/doc/markdown.en.tm
@@ -64,6 +64,30 @@
 
   <section|Features>
 
+  <subsection|Additional <TeXmacs> macros>
+
+  Use the style package <tt|markdown.ts> with <menu|Document|Style|Add
+  package|Markdown|markdown> to enable:
+
+  <\itemize>
+    <item>Support for alternate image formats between <TeXmacs> and markdown
+    via <explain-macro|md-alt-image|tm-image|md-image>. This is useful e.g.
+    to provide SVG and EPS/PDF versions of images for print and web
+    respectively.
+
+    <item>Macros for labels in equation arrays. Using positioning tricks with
+    <explain-macro|htab> results in <LaTeX> code that <name|MathJax> does not
+    support, namely <tt|\\hfill>. For this reason, we provide two macros
+    <explain-macro|eqnarray-lab|lab> and <explain-macro|eqnarray-lab*|lab>
+    which allow positioning of equation labels to the right in equation
+    arrays. The first one also adds a <explain-macro|label|eq:lab> to be used
+    in the document as <explain-macro|reference|eq:lab>, note the prefix
+    <tt|eq:>.
+
+    Symbols (e.g. <math|\<Delta\>> or <math|\<star\>>) are not supported for
+    these labels as of v0.6.1 of the plugin.
+  </itemize>
+
   <subsection|Extensions to vanilla markdown>
 
   Besides standard Markdown, this plugin supports the following:
@@ -90,22 +114,25 @@
   converted from <TeXmacs>, including setting frontmatter values and
   extensions like footnotes and <strike-through|striked through text>.
 
-  Setting values for the frontmatter is suported via a dedicated macro
-  defined in the style package <code*|hugo.ts>. To use it go to
-  <menu|Document|Style|Add package|Markdown|hugo> or use the icon in the
-  focus bar.
-
-  Now you can use the macro <explain-macro|hugo-front> to input any number of
-  <verbatim|key\|value> pairs as arguments, one argument each. That is:
-  insert the tag <explain-macro|hugo-front>, then use structured insert right
-  to add two arguments and use the first for the key and the second for the
-  value. Repeat as needed. It is possible to use strings, booleans
-  (<verbatim|true>, <verbatim|false>) or dates (insert with
-  <explain-macro|date>). Additionally, the macro <explain-macro|pdf-name>
-  inserts the name of the file with <verbatim|.pdf> appended. To enter a
-  list, input <explain-macro|tuple> as the value and use structured insert
-  right to add items. To input a dictionary, use <explain-macro|dict> and
-  again use structured insert to create tuples of <verbatim|key\|value>.
+  Use the style package <tt|hugo.ts> with <menu|Document|Style|Add
+  package|Markdown|hugo> to enable:
+
+  <\itemize>
+    <item>All of the markdown extensions above
+
+    <item>Setting of values for the frontmatter. With
+    <explain-macro|hugo-front> one can input any number of
+    <verbatim|key\|value> pairs as arguments, one argument each. That is:
+    insert the tag <explain-macro|hugo-front>, then use structured insert
+    right to add two arguments and use the first for the key and the second
+    for the value. Repeat as needed. It is possible to use strings, booleans
+    (<verbatim|true>, <verbatim|false>) or dates (insert with
+    <explain-macro|date>). Additionally, the macro <explain-macro|pdf-name>
+    inserts the name of the file with <verbatim|.pdf> appended. To enter a
+    list, input <explain-macro|tuple> as the value and use structured insert
+    right to add items. To input a dictionary, use <explain-macro|dict> and
+    again use structured insert to create tuples of <verbatim|key\|value>.
+  </itemize>
 
   <subsubsection|Supported shortcodes>
 
@@ -144,8 +171,9 @@
     see stack traces and such in case you are running into problems.
 
     <item>EPS and PDF images are not supported by browsers. As a workaround
-    allowing to have both, use <explain-macro|md-alt-image>. The first
-    argument is for <TeXmacs> and the second one for the markdown export.
+    allowing to have both, use <explain-macro|md-alt-image> in package
+    <tt|markdown.ts> as explained above. The first argument is for <TeXmacs>
+    and the second one for the markdown export.
 
     <item>The converter supports only one style. For instance all
     environments have emphasized bodies and bold names,
@@ -161,4 +189,4 @@
   <\collection>
     <associate|markdown-auto-export|../README.md>
   </collection>
-</initial>
+</initial>
diff --git a/packages/markdown/hugo.ts b/packages/markdown/hugo.ts
@@ -3,13 +3,38 @@
 <style|source>
 
 <\body>
+  <active*|<\src-title>
+    <src-package|hugo|0.3>
+
+    <\src-purpose>
+      Helper macros for the Hugo extensions to the markdown plugin
+    </src-purpose>
+
+    <src-copyright|2021|Miguel de Benito Delgado>
+
+    <\src-license>
+      This software falls under the <hlink|GNU general public license,
+      version 3 or later|$TEXMACS_PATH/LICENSE>. It comes WITHOUT ANY
+      WARRANTY WHATSOEVER. You should have received a copy of the license
+      which the software. If not, see <hlink|http://www.gnu.org/licenses/gpl-3.0.html|http://www.gnu.org/licenses/gpl-3.0.html>.
+    </src-license>
+  </src-title>>
+
+  <use-package|markdown>
+
+  <\active*>
+    <\src-comment>
+      Frontmatter utilities
+    </src-comment>
+  </active*>
+
   <assign|hugo-front|<xmacro|args|<flag|frontmatter parameters|orange>>>
 
   <assign|dict|<xmacro|kvs|dict (to do: display keys and values)>>
 
   <assign|pdf-name|<macro|<use-module|(markdown-utils)><extern|download-name>>>
 
-  <assign|md-alt-image|<macro|img|alt-img|<arg|img>>>
+  \;
 </body>
 
 <\initial>

diff --git a/packages/markdown/markdown.ts b/packages/markdown/markdown.ts
@@ -0,0 +1,47 @@
+<TeXmacs|2.1>
+
+<style|source>
+
+<\body>
+  <active*|<\src-title>
+    <src-package|markdown|0.2>
+
+    <\src-purpose>
+      Helper macros for the markdown plugin
+    </src-purpose>
+
+    <src-copyright|2022|Miguel de Benito Delgado>
+
+    <\src-license>
+      This software falls under the <hlink|GNU general public license,
+      version 3 or later|$TEXMACS_PATH/LICENSE>. It comes WITHOUT ANY
+      WARRANTY WHATSOEVER. You should have received a copy of the license
+      which the software. If not, see <hlink|http://www.gnu.org/licenses/gpl-3.0.html|http://www.gnu.org/licenses/gpl-3.0.html>.
+    </src-license>
+  </src-title>>
+
+  -- Alternate images for markdown and <TeXmacs> (first arg is used in
+  markdown)
+
+  <assign|md-alt-image|<macro|img|alt-img|<arg|img>>>
+
+  <\active*>
+    <\src-comment>
+      Labels for equation arrays. Use on the rightmost column to add just
+      text or text and labels to refer to.
+    </src-comment>
+  </active*>
+
+  <assign|eqnarray-lab|<macro|lab|<htab|5mm><set-binding|<arg|lab>><around*|(|<arg|lab>|)><label|<merge|eq:|<arg|lab>>><flag|<merge|eq:|<arg|lab>>|blue>>>
+
+  <assign|eqnarray-lab*|<macro|lab|<htab|5mm><around*|(|<arg|lab>|)>>>
+</body>
+
+<\initial>
+  <\collection>
+    <associate|global-author|Miguel de Benito Delgado>
+    <associate|global-subject|>
+    <associate|global-title|Macros for the TransferLab>
+    <associate|page-medium|paper>
+  </collection>
+</initial>
diff --git a/progs/markdown-smart-ref.scm b/progs/markdown-smart-ref.scm
@@ -45,6 +45,9 @@
     (rem-ref . ("Remark" "identity"))
     (remark-ref . ("Remark" "identity"))
     (note-ref . ("Note" "identity"))
+    (marginal-ref . ("Note" "identity"))
+    (mnote-ref . ("Note" "identity"))
+    (side-ref . ("Note" "identity"))
     (war-ref . ("Warning" "identity"))
     (warning-ref . ("Warning" "identity"))
     (ex-ref . ("Example" "identity"))

diff --git a/progs/markdown-utils.scm b/progs/markdown-utils.scm
@@ -137,7 +137,8 @@
       (begin (display* "Labels must be strings. Received: " s "\n") "")))
 
 (define-public (string-punctuation? s)
-  (not (string-match "\\w+" s)))
+  (with s* (tm-encoding->md-encoding s #t)
+    (not (string-match "\\w+" s*))))
 
 (define-public (string-recompose-space s)
   (string-recompose s " "))
@@ -155,18 +156,22 @@
    (map (cut tm-encoding->md-encoding <> (md-get 'file?))
         (string-split s #\newline))))
 
-(define md-no-first-chars (string->char-set ">-*123456789:"))
+(define-public md-special-line-start-regex
+  (make-regexp "^( *)([-+*>:]|[0123456789]\\.)(.*)$"))
+
+(define (md-special-line-start? s)
+  (with matches (regexp-exec md-special-line-start-regex s)
+    (not (not matches))))  ; HACK: convert to bool
 
 (define (maybe-join match prev)
+  "Whether to join @match to @prev words to avoid lines splitting at @match"
   (with m (match:substring match 0)
-    (if (and (list>0? prev)
-             (> (string-length m) 0)
-             (string-every md-no-first-chars m 0 1))
+    (if (and (list>0? prev) (md-special-line-start? m))
         (cons (string-append (car prev) " " m) (cdr prev))
         (cons m prev))))
 
 (define (safe-split s)
-  "Splits words in @s except when a new word would start with a special char"
+  "Splits @s byw words except when a new word would start with special chars"
   ; HACK: srfi chokes on cork chars, e.g. the backquote \x00 is interpreted
   ; as #\nil, so we need to convert back and forth
   (with s* (tm-encoding->md-encoding s #t)