Conflict with ggplot2 and Arabic & Hindi

[bjcarothers]

A workaround is necessary to get ggplot2 to work with Arabic and Hindi when flextable is loaded. (Issue is also discussed here: https://stackoverflow.com/questions/79324809/conflict-between-flextable-ggplot2-and-arabic-hindi). I narrowed the problem down to textshaping, which is a dependent of ragg, which is a dependent of officer, which is a dependent of flextable. Here's an example that also includes English and Russian to show that things work fine with those:

library(dplyr) # data management
library(flextable) # used elsewhere in application
library(ggplot2) # chart

datDf <- tibble(En = c("Orange","Purple"),
                Ru = c("оранжевый","Пурпурный"),
                Hi = c("नारंगी","जामुनी"),
                Ar = c("برتقالي","بنفسجي"), # R handles the direction change
                xMin = c(1,2),
                xMax = c(3,4)) %>%
  # keep rows in order for plot
  mutate(across(c(En,Ru,Hi,Ar), ~factor(.x, levels=rev(.x)))) 

plotFun <- function(labLang){
  # x-axis labels
  xTiks <- switch(deparse(substitute(labLang)),
                  En = c("1.0\nAlmost Never",
                         "2.0\nRarely",
                         "3.0\nSometimes",
                         "4.0\nOften",
                         "5.0\nAlmost Always"),
                  Ru = c("1.0\nПочти никогда",
                         "2.0\nРедко",
                         "3.0\nИногда",
                         "4.0\nЧасто",
                         "5.0\nПочти всегда"),
                  Hi = c("1.0\nलगभग कभी नही",
                         "2.0\nशायद ही कभी",
                         "3.0\nकभी-कभी",
                         "4.0\nअक्सर",
                         "5.0\nलगभग हमेशा"),
                  Ar = c("1.0\nتقريبًا لا على الإطلاق",
                         "2.0\nنادرًا",
                         "3.0\nفي بعض الأحيان",
                         "4.0\nغالبًا",
                         "5.0\nتقريبًا دائمًا")
                  )
  plt1 <- ggplot(datDf, aes(y={{labLang}}, xmin=xMin, xmax=xMax)) +
    geom_linerange()
  plt2 <- if (deparse(substitute(labLang)) == "Ar") {
    # handle right-to-left for Arabic
    plt1 +
      scale_x_reverse(labels=xTiks, limits=c(5,1)) +
      scale_y_discrete(position="right")
  } else {
    plt1 +
      scale_x_continuous(labels=xTiks, limits=c(1,5))
  }
  return(plt2)
}

plotFun(En) # English
plotFun(Ru) # Russian
plotFun(Hi) # Hindi
plotFun(Ar) # Arabic

English output (works fine):

Russian output (works fine):

Hindi output (text is removed):

The Arabic plot doesn't render at all.

Using unloadNamespace("textshaping") before running the plot functions didn't work, but using pkgload::unload("textshaping") did work, and the plots render as expected.

Hindi:

Arabic:

Hindi and Arabic seem to render fine when actually using them in a flextable (as far as I can tell - I still need to confirm with my translators), so the issue is thus far limited to ggplot2. The pkgload::unload workaround serves me fine at the moment, but thought y'all might like to know something is up.

[thomasp85]

Can you share your operating system and the graphics device you are using

[bjcarothers]

OS: Windows 10 Enterprise V 22H2 build 19045.5247
Graphics device: For this example, I'm knitting HTML from an .Rmd with the default settings, so my understanding is that the device is .png. In my real application, I'm knitting a pagedown PDF, so I'm assuming some variant of .pdf. Results for both are similar.

[thomasp85]

Just as a sanity check, can I get you to see if everything works as expected using the ragg graphics devices?

[bjcarothers]

Using Tools > Global Options > Graphics and selecting AGG for the Backend, then commenting out pkgload::unload("textshaping") I get the following:

For Hindi, the output in the knitted HTML file is the same as above with the missing labels. The inline output looks like this:

For Arabic, it wouldn't render in the knitted HTML file, but the inline output was fine.

[thomasp85]

The setting you mention only affects the plot pane. For knitting you need to set dev="ragg_png" for the chunk option

[bjcarothers]

Keeping the AGG selection on the backend, keeping pkgload::unload("textshaping") commented out, and adding dev="ragg_png" to the chunk option yields the same inline output as the previous step (Arabic ok, Hindi with the grayed rectangles instead of labels), and now the knitted html output aligns with that - Arabic is fine and Hindi with rectangles instead of labels.

[thomasp85]

Ok, hmm. I think there is two things at play. Your original issue I think is a duplicate of #35 which I have yet to find a sensible reason for. I have no idea why attaching textshaping has influence over graphics devices that doesn't use it.

As for the rectangles with hindi it appears that the any fallback isn't found.

What do you get if you run systemfonts::font_fallback() with the hindi text as input?

[bjcarothers]

Running

systemfonts::font_fallback(c("1.0\nलगभग कभी नही",
                         "2.0\nशायद ही कभी",
                         "3.0\nकभी-कभी",
                         "4.0\nअक्सर",
                         "5.0\nलगभग हमेशा"))

yields a 5X2 dataframe:

path index
C:\WINDOWS\Fonts\arial.ttf 0
C:\WINDOWS\Fonts\arial.ttf 0
C:\WINDOWS\Fonts\arial.ttf 0
C:\WINDOWS\Fonts\arial.ttf 0
C:\WINDOWS\Fonts\arial.ttf 0

[thomasp85]

Thanks - that is obviously not good. What is the output of systemfonts::system_fonts()?

[bjcarothers]

I apparently have 458 fonts installed so I won't give you the whole thing, but the first 10 rows are:

path index name family style weight width italic monospace
"C:\WINDOWS\Fonts\arial.ttf" 0 ArialMT Arial Regu… normal norm… FALSE FALSE
"C:\WINDOWS\Fonts\ariblk.ttf" 0 Arial-Black Arial Black heavy norm… FALSE FALSE
"C:\WINDOWS\Fonts\arialbd.ttf" 0 Arial-BoldMT Arial Bold bold norm… FALSE FALSE
"C:\WINDOWS\Fonts\arialbi.ttf" 0 Arial-BoldItalic… Arial Bold… bold norm… TRUE FALSE
"C:\WINDOWS\Fonts\ariali.ttf" 0 Arial-ItalicMT Arial Ital… normal norm… TRUE FALSE
"C:\WINDOWS\Fonts\bahnschrift.ttf" 0 Bahnschrift Bahns… Regu… normal norm… FALSE FALSE
"C:\WINDOWS\Fonts\calibri.ttf" 0 Calibri Calib… Regu… normal norm… FALSE FALSE
"C:\WINDOWS\Fonts\calibrib.ttf" 0 Calibri-Bold Calib… Bold bold norm… FALSE FALSE
"C:\WINDOWS\Fonts\calibriz.ttf" 0 Calibri-BoldItal… Calib… Bold… bold norm… TRUE FALSE
"C:\WINDOWS\Fonts\calibrii.ttf" 0 Calibri-Italic Calib… Ital… normal norm… TRUE FALSE

[thomasp85]

Haha, thanks. It was mainly a test to check whether it came up almost empty...

I'll think some more on this and get back to you

[thomasp85]

If you are able to compile textshaping, can I get you to try to install textshaping#56 (pak::pak("r-lib/textshaping#56")) and see if it fixes your initial issue? (don't try to use it with ragg - it will likely crash)

[bjcarothers]

With the new version of textshaping, the default selection for the backend graphics device, and no options for the code chunk, I get plots with lines and numbers but no labels (aside from "Hi" and "Ar" on the y axis) for both Hindi and Arabic in both the inline and knitted HTML output.

[thomasp85]

thanks - I've dusted off a Win VM and can reproduce locally. Hopefully I can get to the bottom of this