install_local(upgrade="never") doesn't allow to install the golem package-app

[agronomofiorentini]

Hello,
I would like to thank again the ThinkR team for creating this package making the development of a web application fast and solid.

I have finished to develop the app and i am ready to deploy it. In this case i would use the docker system.

The follow code is the dockerfile

FROM rocker/r-ver:4.0.5
RUN apt-get update && apt-get install -y  gdal-bin git-core imagemagick libcurl4-openssl-dev libgdal-dev libgeos-dev libgeos++-dev libgit2-dev libicu-dev libpng-dev libpq-dev libproj-dev libssl-dev libudunits2-dev libxml2-dev make pandoc pandoc-citeproc && rm -rf /var/lib/apt/lists/*
RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" >> /usr/local/lib/R/etc/Rprofile.site
RUN R -e 'install.packages("remotes")'
RUN Rscript -e 'remotes::install_version("glue",upgrade="never", version = "1.4.2")'
RUN Rscript -e 'remotes::install_version("processx",upgrade="never", version = "3.5.2")'
RUN Rscript -e 'remotes::install_version("stringr",upgrade="never", version = "1.4.0")'
RUN Rscript -e 'remotes::install_version("htmltools",upgrade="never", version = "0.5.1.1")'
RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "1.0-2")'
RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.6.0")'
RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.4.1")'
RUN Rscript -e 'remotes::install_version("pkgload",upgrade="never", version = "1.2.1")'
RUN Rscript -e 'remotes::install_version("attempt",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("rgeos",upgrade="never", version = "0.5-7")'
RUN Rscript -e 'remotes::install_version("RPostgreSQL",upgrade="never", version = "0.6-2")'
RUN Rscript -e 'remotes::install_version("leaflet.extras",upgrade="never", version = "1.0.0")'
RUN Rscript -e 'remotes::install_version("testthat",upgrade="never", version = "3.0.2")'
RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("spelling",upgrade="never", version = "2.2")'
RUN Rscript -e 'remotes::install_version("shinyvalidate",upgrade="never", version = "0.1.1")'
RUN Rscript -e 'remotes::install_version("shinyjs",upgrade="never", version = "2.0.0")'
RUN Rscript -e 'remotes::install_version("shinyalert",upgrade="never", version = "2.0.0")'
RUN Rscript -e 'remotes::install_version("RPostgres",upgrade="never", version = "1.3.2")'
RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.18")'
RUN Rscript -e 'remotes::install_version("cicerone",upgrade="never", version = "1.0.4")'
RUN Rscript -e 'remotes::install_version("blastula",upgrade="never", version = "0.3.2")'
RUN Rscript -e 'remotes::install_github("r-lib/rlang@dc03e4473000d0424dacd4c8cf08211b7fb8bc97")'
RUN Rscript -e 'remotes::install_github("r-spatial/mapview@8d927d4eded65f1528bef7e6cdd2e83bd83fbaee")'
RUN Rscript -e 'remotes::install_github("JohnCoene/waiter@ce5fdaa1fc1224ba657c0906c7900b275cfee1d7")'
RUN Rscript -e 'remotes::install_github("mablab/rpostgis@74a6ac71bcc21b7b5edf0ed6b59dcbfd4d7bfc59")'
RUN Rscript -e 'remotes::install_github("r-spatial/mapedit@8328411a45513f788dc25e6b4214df53363d6494")'
RUN Rscript -e 'remotes::install_github("RinteRface/bs4Dash@93c7c45e2a3f43e7a1c71ce375dea0397e8ff3fa")'
RUN mkdir /build_zone
ADD . /build_zone
WORKDIR /build_zone
RUN R -e 'remotes::install_local(upgrade="never")'
RUN R -e 'remotes::install_local(upgrade="never")'
RUN rm -rf /build_zone
EXPOSE 80
CMD R -e "options('shiny.port'=80,shiny.host='0.0.0.0');ReportSatellite::run_app()"

As reported here #739 i have duplicated this line "RUN R -e 'remotes::install_local(upgrade="never")'"

Then by using the terminal i go the the folder of the web app using cd path/to/folder

Then i have used the follow comands

docker build -t ciaone .

Then when the image it was created i used the next command

docker run ciaone

but with no success, getting always the same error output which is

Error in loadNamespace(name) :
there is no package called ‘ReportSatellite’
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

It seems that the package-app it is not installed yet in the docker environment.

Could you help me to solve this?

Thanks

[agronomofiorentini]

Hi again Colin,

I tried to modify the above code at the line

RUN R -e 'remotes::install_local(upgrade="never")'
RUN R -e 'remotes::install_local(upgrade="never")'

I have tried to to what was reported here https://github.com/ThinkR-open/golem/issues/730
But didn't work.

Then i tried to modify the previouse lines with the follow lines

RUN R -e 'remotes::install_local(path="~Desktop/webapp_0.0.0.9000.tar.gz", upgrade="never")'
RUN R -e 'remotes::install_local(path="~Desktop/webapp_0.0.0.9000.tar.gz", upgrade="never")'

But it still not working.

Could you help me to solve this?

Thanks.
Marco

[tottidici]

Hi !
I have exactly the same problem
I started a discussion ThinkR-open/golem#760

David

[agronomofiorentini]

Hi @tottidici,
This morning I tried to develop with docker the default app that is generated when you launch a golem project and everything worked correctly.

Then I added some data using the use_raw_data() function and then I inserted a simple plot.

I have checked the DESCRIPTION file and the LazyData: true is present.

Already at this point the docker system gives me the same error as reported here in advance.

Error in loadNamespace(name) :
there is no package called ‘ReportSatellite’
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

While when i am using run_dev all works perfectly.

I don't know how to continue at this point.

[tottidici]

so the packages would be involved?

[agronomofiorentini]

I actually don't know if the packages would be involved

[VincentGuyader]

Hi

I need to have the log of the

docker build -t ciaone .

To see why you cant install your package from source.

What is the check output ?

[agronomofiorentini]

Hi @VincentGuyader,
Thanks to help us.

I have created the docker file with golem::add_dockerfile().

Then from the terminal i went to the folder path by using cd path/to/folder

Then i execute the follow command docker build -t ciaone .

And now gave me the follow error in the remotes::install_local(upgrade="never")

=> => # begin installing package ‘terra’
=> => # begin installing package ‘systemfonts’
=> => # make: *** [Makefile:9: systemfonts.ts] Error 1

[ColinFay]

Hey y'all,

As far as I see it the current issue is that the docker building process doesn't exit correctly when the installation of the app fails, so you end up with a successful build of the docker image but with the package for the app not installed.
My wild guess is that {remotes} doesn't throw the correct exit code and/or we need to configure the dockerfile to fail correctly.

[VincentGuyader]

@tottidici and @agronomofiorentini can you share (privately ? ) your whole project with Colin and/or me ?

90% of chance that the issue is in your package, but in that case, docker build should have returned an error, so this seems to be a {dockerfiler} issue too ( because of {remotes})

[agronomofiorentini]

For me no problem, indeed it is a pleasure to understand where the problem is.

How can i share the whole project? send email to [email protected]?

[agronomofiorentini]

golemDocker.zip

[ColinFay]

I think it's related to #9

[agronomofiorentini]

I have add options(warn = 2) to the remotes::install_local(upgrade="never") and i have tried again to build the image

This is the output

 > [21/22] RUN R -e 'options(warn = 2);remotes::install_local(upgrade="never")':
#26 0.767
#26 0.767 R version 4.0.5 (2021-03-31) -- "Shake and Throw"
#26 0.767 Copyright (C) 2021 The R Foundation for Statistical Computing
#26 0.767 Platform: x86_64-pc-linux-gnu (64-bit)
#26 0.767
#26 0.767 R is free software and comes with ABSOLUTELY NO WARRANTY.
#26 0.767 You are welcome to redistribute it under certain conditions.
#26 0.767 Type 'license()' or 'licence()' for distribution details.
#26 0.767
#26 0.767 R is a collaborative project with many contributors.
#26 0.767 Type 'contributors()' for more information and
#26 0.767 'citation()' on how to cite R or R packages in publications.
#26 0.767
#26 0.767 Type 'demo()' for some demos, 'help()' for on-line help, or
#26 0.767 'help.start()' for an HTML browser interface to help.
#26 0.767 Type 'q()' to quit R.
#26 0.767
#26 0.892 > options(warn = 2);remotes::install_local(upgrade="never")
#26 4.209 Installing 4 packages: systemfonts, svglite, leafpop, mapview
#26 4.210 Installing packages into ‘/usr/local/lib/R/site-library’
#26 4.210 (as ‘lib’ is unspecified)
#26 4.991 trying URL 'https://cran.rstudio.com/src/contrib/systemfonts_1.0.2.tar.gz'
#26 5.497 Content type 'application/x-gzip' length 80693 bytes (78 KB)
#26 5.497 ==================================================
#26 5.527 downloaded 78 KB
#26 5.527
#26 5.530 trying URL 'https://cran.rstudio.com/src/contrib/svglite_2.0.0.tar.gz'
#26 5.981 Content type 'application/x-gzip' length 122260 bytes (119 KB)
#26 5.981 ==================================================
#26 6.020 downloaded 119 KB
#26 6.020
#26 6.023 trying URL 'https://cran.rstudio.com/src/contrib/leafpop_0.1.0.tar.gz'
#26 6.326 Content type 'application/x-gzip' length 1870372 bytes (1.8 MB)
#26 6.329 ==================================================
#26 6.543 downloaded 1.8 MB
#26 6.543
#26 6.546 trying URL 'https://cran.rstudio.com/src/contrib/mapview_2.10.0.tar.gz'
#26 6.950 Content type 'application/x-gzip' length 1683417 bytes (1.6 MB)
#26 6.953 ==================================================
#26 7.216 downloaded 1.6 MB
#26 7.216
#26 8.254 begin installing package ‘systemfonts’
#26 8.763 make: *** [Makefile:4: systemfonts.ts] Error 1
#26 8.763 make: Target 'all' not remade because of errors.
#26 8.765 * installing *source* package ‘systemfonts’ ...
#26 8.765 ** package ‘systemfonts’ successfully unpacked and MD5 sums checked
#26 8.765 ** using staged installation
#26 8.765 Package fontconfig was not found in the pkg-config search path.
#26 8.765 Perhaps you should add the directory containing `fontconfig.pc'
#26 8.765 to the PKG_CONFIG_PATH environment variable
#26 8.765 No package 'fontconfig' found
#26 8.765 Package freetype2 was not found in the pkg-config search path.
#26 8.765 Perhaps you should add the directory containing `freetype2.pc'
#26 8.765 to the PKG_CONFIG_PATH environment variable
#26 8.765 No package 'freetype2' found
#26 8.765 Using PKG_CFLAGS=
#26 8.765 Using PKG_LIBS=-lfontconfig -lfreetype
#26 8.765 --------------------------- [ANTICONF] --------------------------------
#26 8.765 Configuration failed to find the fontconfig freetype2 library. Try installing:
#26 8.765  * deb: libfontconfig1-dev (Debian, Ubuntu, etc)
#26 8.765  * rpm: fontconfig-devel (Fedora, EPEL)
#26 8.765  * csw: fontconfig_dev (Solaris)
#26 8.765  * brew: freetype (OSX)
#26 8.765 If fontconfig freetype2 is already installed, check that 'pkg-config' is in your
#26 8.765 PATH and PKG_CONFIG_PATH contains a fontconfig freetype2.pc file. If pkg-config
#26 8.765 is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
#26 8.765 R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
#26 8.765 -------------------------- [ERROR MESSAGE] ---------------------------
#26 8.765 <stdin>:1:10: fatal error: fontconfig/fontconfig.h: No such file or directory
#26 8.765 compilation terminated.
#26 8.765 --------------------------------------------------------------------
#26 8.765 ERROR: configuration failed for package ‘systemfonts’
#26 8.765 * removing ‘/usr/local/lib/R/site-library/systemfonts’
#26 8.766 cat: svglite.out: No such file or directory
#26 8.767 cat: leafpop.out: No such file or directory
#26 8.769 cat: mapview.out: No such file or directory
#26 8.772 Error: Failed to install 'golemDocker' from local:
#26 8.772   (converted from warning) installation of one or more packages failed,
#26 8.772   probably ‘systemfonts’, ‘svglite’, ‘leafpop’, ‘mapview’
#26 8.772 Execution halted
------
executor failed running [/bin/sh -c R -e 'options(warn = 2);remotes::install_local(upgrade="never")']: exit code: 1

[tottidici]

try this ThinkR-open/golem#760, it's ok for me

[agronomofiorentini]

@tottidici thanks to report your news about your issue.

But i still have the problem to use docker.

And from what i understand the problem is about this systemfonts package, which i didn't mention it in my app and i didn't add it to the description file.

#42 10.66 ERROR: configuration failed for package ‘systemfonts’

This time i will try to install it and then rebuild the docker image

[tottidici]

@agronomofiorentini
I have created a project golem with package ‘systemfonts’
then i have created dockerfile and i have added this
RUN echo "options(warn = 2);" >> $R_HOME/etc/Rprofile.site for debug
So, this error appeared :

------
 > [ 8/14] RUN Rscript -e 'remotes::install_version("systemfonts",upgrade="never", version = "1.0.3")':
#11 2.408 Downloading package from url: https://cran.rstudio.com/src/contrib/systemfonts_1.0.3.tar.gz
#11 4.160 Installing 1 packages: cpp11
#11 4.162 Installing package into ‘/usr/local/lib/R/site-library’
#11 4.162 (as ‘lib’ is unspecified)
#11 5.362 trying URL 'https://cran.rstudio.com/src/contrib/cpp11_0.4.0.tar.gz'
#11 5.667 Content type 'application/x-gzip' length 317563 bytes (310 KB)
#11 5.667 ==================================================
#11 5.723 downloaded 310 KB
#11 5.723
#11 6.338 * installing *source* package ‘cpp11’ ...
#11 6.348 ** package ‘cpp11’ successfully unpacked and MD5 sums checked
#11 6.349 ** using staged installation
#11 6.376 ** R
#11 6.388 ** inst
#11 6.397 ** byte-compile and prepare package for lazy loading
#11 7.216 ** help
#11 7.248 *** installing help indices
#11 7.283 ** building package indices
#11 7.673 ** installing vignettes
#11 7.689 ** testing if installed package can be loaded from temporary location
#11 8.055 ** testing if installed package can be loaded from final location
#11 8.425 ** testing if installed package keeps a record of temporary installation path
#11 8.429 * DONE (cpp11)
#11 8.442
#11 8.442 The downloaded source packages are in
#11 8.442       ‘/tmp/RtmpjShVSO/downloaded_packages’
#11 8.443 Installing package into ‘/usr/local/lib/R/site-library’
#11 8.443 (as ‘lib’ is unspecified)
#11 8.837 * installing *source* package ‘systemfonts’ ...
#11 8.843 ** package ‘systemfonts’ successfully unpacked and MD5 sums checked
#11 8.844 ** using staged installation
#11 8.851 Using PKG_CFLAGS=
#11 8.851 Using PKG_LIBS=-lfontconfig -lfreetype
#11 9.016 --------------------------- [ANTICONF] --------------------------------
#11 9.016 Configuration failed to find the fontconfig freetype2 library. Try installing:
#11 9.016  * deb: libfontconfig1-dev (Debian, Ubuntu, etc)
#11 9.016  * rpm: fontconfig-devel (Fedora, EPEL)
#11 9.016  * csw: fontconfig_dev (Solaris)
#11 9.016  * brew: freetype (OSX)
#11 9.016 If fontconfig freetype2 is already installed, check that 'pkg-config' is in your
#11 9.016 PATH and PKG_CONFIG_PATH contains a fontconfig freetype2.pc file. If pkg-config
#11 9.016 is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
#11 9.016 R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
#11 9.016 -------------------------- [ERROR MESSAGE] ---------------------------
#11 9.018 <stdin>:1:10: fatal error: fontconfig/fontconfig.h: No such file or directory
#11 9.018 compilation terminated.
#11 9.018 --------------------------------------------------------------------
#11 9.020 ERROR: configuration failed for package ‘systemfonts’#11 9.020 * removing ‘/usr/local/lib/R/site-library/systemfonts’
#11 9.036 Error: Failed to install 'unknown package' from URL:
#11 9.036   (converted from warning) installation of package ‘/tmp/RtmpjShVSO/remotes8525db6c3/systemfonts’ had non-zero exit status
#11 9.036 Execution halted

So, I have added this in dockerfile libfontconfig1-dev like that
RUN apt-get update && apt-get install -y git-core libcurl4-openssl-dev libfontconfig1-dev libgit2-dev libicu-dev libssl-dev libxml2-dev make pandoc pandoc-citeproc && rm -rf /var/lib/apt/lists/*

and it's ok !

@agronomofiorentini I hope this helps you

[agronomofiorentini]

@tottidici, it is working now.

Thanks a lot to you and the thinkR team.

[tottidici]

so nice !!!!!!!!
I'm very happy to have helped you, especially since I'm not a developer and I've never had any training on docker.
In fact, you have to read the debugging when you build the container.
Thanks to Vincent Guyader for making me add this line in the docker file, I didn't understand immediately what it was for, but it is very useful!

Maybe we should create an issue on the golem project, to improve the construction of the dockerfile, so that the linux libraries necessary to the functioning of the R packages are integrated automatically (if it is possible)
@ColinFay what do you think about this ?

Merci à tous !!

[agronomofiorentini]

Hi again to everyone,
I finally managed to develop my app successfully with docker thanks to this discussion and people that participated.

Now I have a new problem.

When using the app the user fills out a form and it is sent to a postgres database, I get an error and it is as follow

Warning: Error in : could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

I report here below as usual the dockerfile where I added EXPOSE 5432 to solve the problem but without success

FROM rocker/r-ver:4.0.5
RUN apt-get update && apt-get install -y  gdal-bin git-core imagemagick libcurl4-openssl-dev libfontconfig1-dev libgdal-dev libgeos-dev libgeos++-dev libgit2-dev libicu-dev libpng-dev libpq-dev libproj-dev libssl-dev libudunits2-dev libxml2-dev make pandoc pandoc-citeproc && rm -rf /var/lib/apt/lists/*
RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" >> /usr/local/lib/R/etc/Rprofile.site
RUN R -e 'install.packages("remotes")'
RUN Rscript -e 'remotes::install_version("glue",upgrade="never", version = "1.4.2")'
RUN Rscript -e 'remotes::install_version("processx",upgrade="never", version = "3.5.2")'
RUN Rscript -e 'remotes::install_version("stringr",upgrade="never", version = "1.4.0")'
RUN Rscript -e 'remotes::install_version("htmltools",upgrade="never", version = "0.5.2")'
RUN Rscript -e 'remotes::install_version("systemfonts",upgrade="never", version = "1.0.3")'
RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "1.0-3")'
RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.7.1")'
RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.4.1")'
RUN Rscript -e 'remotes::install_version("pkgload",upgrade="never", version = "1.2.2")'
RUN Rscript -e 'remotes::install_version("attempt",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("rgeos",upgrade="never", version = "0.5-8")'
RUN Rscript -e 'remotes::install_version("RPostgreSQL",upgrade="never", version = "0.7-2")'
RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")'
RUN Rscript -e 'remotes::install_version("leaflet.extras",upgrade="never", version = "1.0.0")'
RUN Rscript -e 'remotes::install_version("testthat",upgrade="never", version = "3.1.0")'
RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("spelling",upgrade="never", version = "2.2")'
RUN Rscript -e 'remotes::install_version("shinyvalidate",upgrade="never", version = "0.1.1")'
RUN Rscript -e 'remotes::install_version("shinyjs",upgrade="never", version = "2.0.0")'
RUN Rscript -e 'remotes::install_version("shinyalert",upgrade="never", version = "2.0.0")'
RUN Rscript -e 'remotes::install_version("RPostgres",upgrade="never", version = "1.4.1")'
RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.19")'
RUN Rscript -e 'remotes::install_version("cicerone",upgrade="never", version = "1.0.4")'
RUN Rscript -e 'remotes::install_version("bs4Dash",upgrade="never", version = "2.0.3")'
RUN Rscript -e 'remotes::install_version("blastula",upgrade="never", version = "0.3.2")'
RUN Rscript -e 'remotes::install_github("r-lib/rlang@dc03e4473000d0424dacd4c8cf08211b7fb8bc97")'
RUN Rscript -e 'remotes::install_github("JohnCoene/waiter@ce5fdaa1fc1224ba657c0906c7900b275cfee1d7")'
RUN Rscript -e 'remotes::install_github("mablab/rpostgis@74a6ac71bcc21b7b5edf0ed6b59dcbfd4d7bfc59")'
RUN Rscript -e 'remotes::install_github("r-spatial/mapedit@8328411a45513f788dc25e6b4214df53363d6494")'
RUN mkdir /build_zone
ADD . /build_zone
WORKDIR /build_zone
RUN R -e 'options(warn = 2);remotes::install_local(upgrade="never")'
RUN rm -rf /build_zone
EXPOSE 5432
EXPOSE 80
CMD R -e "options('shiny.port'=80,shiny.host='0.0.0.0');ReportSatellite::run_app()"

Aas always i kindly ask you to help me because i have little experience on shinyapp development using docker, but i'm appreciating its capabilities

[tottidici]

Hi !

I think you only need one port, only your docker's port
The postgres connection port must be in the connection string in R like this

dbConnect(dbDriver("PostgreSQL"),user="postgres",password="****", dbname="madb",host="192.168.1.2",port=5432

[agronomofiorentini]

hello everyone again
I still haven't managed to solve this problem.

That is, I can now create the image in docker, but when I try to connect this image with a remote postgresql database it crashes.

I get the following error.

Warning: Error in : could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

I report here below the whole project, and I would kindly ask you to help me to solve this last problem.
DockerApp.zip

[agronomofiorentini]

I just solved this.
i just remove the last line of the docker file which remove all the content of the buildzone directory.
Now it is working perfectly.
From my side it can be close this issue.

I want to thank every participant