UTF-8 related bug in verbatimTextOutput?

[vnijs]

The string "папка0" (Russian for "folder0") prints out fine in Rstudio and Shiny on macOS. However, on Windows it only prints OK in RStudio but not in Shiny verbatimTextOuput. See below code and screenshot below. I checked that the encoding of the string is UTF-8. Is this a bug in shiny?

FYI: Also posted @ https://community.rstudio.com/t/windows-encoding-utf-8-in-shiny-verbatimtextouptut/11329

library(shiny)

ui <- fluidPage(
  actionButton("prn", "Print"),
  verbatimTextOutput("prn")
)

server <- function(input, output) {
  observeEvent(input$prn, {
    print("папка0")
  })
  
  output$prn <- renderPrint({
    print(input$prn)
    print("папка0")
  })
}

# Run the application 
shinyApp(ui = ui, server = server)

Session Info:

> devtools::session_info()
Session info ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.5.1 (2018-07-02)
 system   x86_64, mingw32             
 ui       RStudio (1.2.679)           
 language (EN)                        
 collate  English_United States.1252  
 tz       America/Los_Angeles         
 date     2018-07-20                  

Packages ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 package    * version    date       source        
 base       * 3.5.1      2018-07-02 local         
 compiler     3.5.1      2018-07-02 local         
 datasets   * 3.5.1      2018-07-02 local         
 devtools     1.13.6     2018-06-27 CRAN (R 3.5.0)
 digest       0.6.15     2018-01-28 CRAN (R 3.5.0)
 graphics   * 3.5.1      2018-07-02 local         
 grDevices  * 3.5.1      2018-07-02 local         
 htmltools    0.3.6      2017-04-28 CRAN (R 3.5.0)
 httpuv       1.4.4.2    2018-07-02 CRAN (R 3.5.1)
 jsonlite     1.5        2017-06-01 CRAN (R 3.5.0)
 later        0.7.3      2018-06-08 CRAN (R 3.5.0)
 magrittr     1.5        2014-11-22 CRAN (R 3.5.0)
 memoise      1.1.0      2017-04-21 CRAN (R 3.5.0)
 methods    * 3.5.1      2018-07-02 local         
 mime         0.5        2016-07-07 CRAN (R 3.5.0)
 promises     1.0.1      2018-04-13 CRAN (R 3.5.0)
 R6           2.2.2      2017-06-17 CRAN (R 3.5.0)
 Rcpp         0.12.17    2018-05-18 CRAN (R 3.5.0)
 rlang        0.2.1      2018-05-30 CRAN (R 3.5.0)
 rstudioapi   0.7.0-9001 2018-05-25 local         
 shiny      * 1.1.0      2018-05-17 CRAN (R 3.5.0)
 stats      * 3.5.1      2018-07-02 local         
 tools        3.5.1      2018-07-02 local         
 utils      * 3.5.1      2018-07-02 local         
 withr        2.1.2      2018-03-15 CRAN (R 3.5.0)
 xtable       1.8-2      2016-02-05 CRAN (R 3.5.0)

[tmastny]

Hi, thanks for the question. Have you tried saving your text files with encoding? Here's an article all about UTF-8 on Windows with Shiny:
https://shiny.rstudio.com/articles/unicode.html

Let me know if that helps.

[vnijs]

What text file are you referring to here @tmastny? app.R? That was already saved as UTF-8 so I don't think that is the issue. Rstudio console prints the character just fine but shiny does not as shown in the screen shot.

[alandipert]

I was able to reproduce this on Windows 10. The culprit appears to be utils::capture.output, a base R function used in renderPrint. Here's a simplified way to repro:

> utils::capture.output(print("папка0"))
[1] "[1] \"<U+043F><U+0430><U+043F><U+043A><U+0430>0\""

Unfortunately it doesn't appear to be a problem we can easily fix. According to @krlmlr, UTF-8 characters outside the current locale don't survive a roundtrip via R's connection interface. Since sink seems like the only way to do something like what capture.output does, I can't think of a workaround.

This is a related issue with the same cause: r-lib/evaluate#59

[jcheng5]

I was going to say you can work around this by using renderText instead of renderPrint, but it looks like that isn't an option either; renderText uses capture.output(cat(value)) to (I'm guessing?) coerce value to a string.

[jcheng5]

I'm moving this to the 1.3 milestone. At best maybe we can detect cases where capture.output isn't necessary and return the UTF-8 encoded string instead.