Unable to get environment variables

[akshaybabloo]

By default, Nginx Unit is not able to get the environment variables unless they are set through environment. Is there a way to get the environment variable from the system environment?

[i4ki]

@akshaybabloo What application language are you using?

if you use python, just import package "os".

import os

def application(environ, start_fn):
    start_fn('200 OK', [('Content-Type', 'text/plain')])
    return [os.environ["PATH"]]

same thing for Go, using os.Getenv(name) works.

[VBart]

Note that for PHP in order to have access to environment, the variables_order PHP configuration option set in php.ini (or via Unit configuration) must include E. See the PHP documentation for details: https://www.php.net/manual/en/ini.core.php#ini.variables-order

[akshaybabloo]

I am using Python and Django, when I use os.environ["PATH"] I get KeyError

[akshaybabloo]

Also, I am using PowerShell Core in Ubuntu 18.04

[i4ki]

I just installed PowerShell to test this, and it works here.

PS /vagrant/src/unit> ./build/unitd
2019/08/29 10:11:46.911 [info] 4811#4811 unit started
curl --unix-socket /home/vagrant/control.unit.sock -XPUT http://localhost/config/applications/helloworld2 --data-binary @app-python.json
{
	"success": "Reconfiguration done."
}
PS /vagrant/src/unit> curl --unix-socket /home/vagrant/control.unit.sock -XPUT 'http://localhost/config/listeners/*:8888' --data-binary '@listener.json'
{
	"success": "Reconfiguration done."
}
PS /vagrant/src/unit> curl localhost:8888        
/opt/microsoft/powershell/6:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin

Try the following:

PS /vagrant/src/unit> python2
Python 2.7.15+ (default, Nov 27 2018, 23:36:35) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ["PATH"]
'/opt/microsoft/powershell/6:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin'
>>> 

If this doesn't works maybe it's something wrong on your setup.

[VBart]

@akshaybabloo How do you start unitd?

[akshaybabloo]

This is what I have in my docker file

unitd
touch /var/log/access.log
curl -X PUT --data-binary @nginx-unit-config.json --unix-socket /var/run/control.unit.sock http://localhost/config/
tail -f /var/log/access.log -f /var/log/unit.log

[VBart]

@akshaybabloo And where you have set the environment variables for the docker?

[akshaybabloo]

The above code is from Docker-Entrypoint.ps1 which looks like:

$Env:DANGO_SECRET_KEY = "<key>"

Import-Module ./Some-More-Keys.psm1  # This file has more $Env:Key="Value"

# ....

unitd
touch /var/log/access.log
curl -X PUT --data-binary @nginx-unit-config.json --unix-socket /var/run/control.unit.sock http://localhost/config/
tail -f /var/log/access.log -f /var/log/unit.log

[VBart]

@akshaybabloo Please make sure that you have access to these environment variables without Unit, e.g. by running a simple Python script instead of unitd.

[akshaybabloo]

I am able to. Before using Nginx Unit I was running gunicorn and I never had an issue with that

[dannmartens]

I have the same issue.

I am evaluating Nginx Unit as a replacement for Gunicorn as the WSGI server for a Flask application and from what I can observe it seems the system environment is not available (through Python's os API). The Flask application has been configured to exit, when required environment variables are not defined. As a result, configuring Nginx Unit fails immediately (with "error": "Failed to apply new configuration."). In the Unit logs, I see the Flask application balking at the missing environment variables.

We use the common pattern to convey configuration to the application through environment variables—in the end we run it as a Docker container with environment settings.

I haven't tried to Dockerize the app yet, but if setting the Unit config will be necessary as part of the build process, I don't see how this can work. A working config environment will only be available at the time we run the image.

[i4ki]

Hi @dannmartens

How are you trying to set the environment variables?

[i4ki]

Do you mean unitd is not having access to the environment variables set by docker?
Or are you trying to set the environment variables while unitd is running?

[dannmartens]

Hi @tiago4orion,

It's not surprising that in my dev set-up, the environment of the dev user is not available to unitd under systemd. For that to work, I'll need to create a config file from a template and a script which derives the actual config file by replacing the needed environment variables— then curl it in purposely to match this dev environment.

The process model is completely different in our current Docker image, which starts the WSGI server (Waitress, Gunicorn, Netius, ...) directly as the entrypoint.
ENTRYPOINT ["/usr/local/bin/gunicorn", "--config", "config.py", "my_app.wsgi"]
The result is that it works the same in a dev environment as it would in a Docker container. If the config is not available from the environment, the app just exits.

Environment which is passed on via -e with Docker is available to both the root, as well as any other unprivileged user. I haven't Dockerized the app yet with unitd, but a unitd application under systemd might be able to see those.

When I run unitd directly in the dev environment, unprivileged, the Python3 application does see the environment.
unitd --no-daemon --control 127.0.0.1:4444 --pid ~/var/run/unit.pid --log ~/var/log/unit.log
To mimic the Gunicorn approach, I'd consider something like:
ENTRYPOINT ["/usr/sbin/unitd", ...]

But now I can't configure the application, unless I use a second terminal to curl it in. What's the best practice for this kind of use case? Perhaps unitd is not intended to be used like this?

[i4ki]

@dannmartens Take a look here: https://unit.nginx.org/installation/#initial-configuration

By using the official Unit image, you can drop configurations inside the /docker-entrypoint.d/ directory. The configs there are loaded during unit startup.

Let me know if this solves your problem.

[djv57]

I have issue with mith my Flask app on Unit and I suspect that the environment variables are the reason of it. The error in unit.log
AttributeError: 'NoneType' object has no attribute 'drivername'

Environment variables are in .env file (I use python-dotenv also). Database related variable looks like
DATABASE_URL=postgresql://user:password@host_name/database_name

same error appears in case of
DATABASE_URL="postgresql://user:password@host_name/database_name"

[p4block]

Hi, I'm testing the use of Unit to run our Symfony app and despite everything looking good in local dev, upon push to the dev server (which is a container setup configured via massive amounts of env vars) our app defaulted to .env file placeholder values.

I saw some threads and info around and set variables_order = EGPCS in our conf.d/app.ini, overriding the php container's default php.ini of GPCS, which in itself overrides EGPCS from upstream PHP apparently.

I've glanced over what's passed to the container and everything seems normal / the same it was on our previous fcgi container. If this can have unintended consequences I'll be glad to be corrected. I now also wonder how it worked on the fpm container.

Passing the pile of 50+ env vars to the container by using envsubst to poppulate a massive unit.json file was the alternative solution if this didn't work, but that would be incredibly annoying for what should be a simple thing to do.
Starting to think many people have EGPCS by default and are not being exposed to the problem, or that EGPCS is actually the way Unit "expects" you to run the app.

Eagerly awaiting comments on the topic from other people

[andrey-zelenkov]

Hi @p4block,

Thank you for reaching out with your issue. To help diagnose and potentially resolve this problem, we would need a minimal use case that can reproduce the issue.

Could you please provide us with the following information:

The relevant portion of your Symfony configuration related to environment variables, including any code that reads or sets these values.
A simplified example of the environment variables you're using in your Symfony app, preferably with placeholder values for sensitive information.
Steps you've taken to push your app to the dev server, as well as how you're configuring the environment variables in the container.
Any relevant log files, error messages, or output that you've observed on the dev server, if any.

Thank you in advance.

[p4block]

@andrey-zelenkov Don't worry about it, as far as I know there is no issue just a lack of documentation and difference in behavior. I've dialed it down to a functional difference between fpm and unit,
I encountered this issue because I followed best practices of the PHP docs, moving one of the disabled php.ini-$environment shipped by upstream php to php.ini path, where it's actually loaded by default.

❯ docker run --rm -it unit:php bash
root@951b231b1b48:/# ls /usr/local/etc/php/
conf.d	php.ini-development  php.ini-production

Those config files set variables_order = GPCS, which is not a problem in a regular fpm container, but it is on Unit if you want all your container's env vars to be passed to the app blindly. I don't think there's much more to it, and I find the behavior on Unit to make more sense actually.

I'll post my things for reference by other people:

unit.json

{
    "listeners": {
        "*:80": {
            "pass": "routes",
            "forwarded": {
                "client_ip": "X-Forwarded-For",
                "protocol": "X-Forwarded-Proto",
                "recursive": true,
                "source": [
                    "10.0.0.0/8",
                    "172.16.0.0/12",
                    "192.168.0.0/16"
                ]
            }
        }
    },

    "routes": [
        {
            "match": {
                "uri": [
                    "*.php",
                    "*.php/*"
                ]
            },

            "action": {
                "pass": "applications/symfony/direct"
            }
        },
        {
            "action": {
                "share": "/www/public$uri",
                "fallback": {
                    "pass": "applications/symfony/index"
                }
            }
        }
    ],

    "applications": {
        "symfony": {
            "type": "php",
            "limits": {
                "timeout": 90
            },
            "processes": 20,
            "targets": {
                "direct": {
                    "root": "/www/public/"
                },

                "index": {
                    "root": "/www/public/",
                    "script": "index.php"
                }
            }
        }
    }
}

Dockerfile

FROM unit:1.31.0-php8.2 as base 

WORKDIR /www

# Magic PHP extension installer is the best thing since sliced bread
COPY --link --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/

# Note: avoiding a lengthy gd compilation skipping AV1 support 
RUN IPE_GD_WITHOUTAVIF=1 install-php-extensions \
    @composer \
    opcache intl zip gd imagick igbinary msgpack redis memcached pdo_mysql excimer 

# Copy shared PHP configuration to increase RAM
COPY --link docker/app.ini $PHP_INI_DIR/conf.d/

FROM base as prod
ENV APP_ENV prod
ENV COMPOSER_ALLOW_SUPERUSER=1

# Install project dependencies
COPY --link composer.* symfony.* ./
RUN composer install --no-dev --no-cache --no-interaction --no-autoloader --no-scripts

# Copy main app
COPY --link . .

# Generate autoloader
RUN composer dump-autoload --classmap-authoritative

# Generate important folders and content
RUN mkdir -p public var && chown -R unit:unit public var
USER unit
RUN composer run-script post-install-cmd
USER root

# Only use env vars in prod, ignore .env
RUN composer dump-env prod --empty 

# Compile the admin frontend with node
# We actually only define devdependencies for npm, avoiding npm ci --omit=dev
# Makes sense as node doesn't actually run in production and is only used to build
# We do it on a single layer to save space on the final image
# Don't even try to switch to Bun yet. It won't finish the build.
RUN apt-get update && apt-get install -y gnupg \
    && mkdir -p /etc/apt/keyrings \
    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list \
    && apt-get update \
    && apt-get install nodejs -y \
    && npm ci \
    && npm run build \
    && rm -rf node_modules \
    && rm -rf /root/.npm \
    && apt-get autoremove --purge -y nodejs gnupg\
    && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/*.list

# Finally, add web server configuration
COPY --link ./docker/unit.json /docker-entrypoint.d/unit.json
COPY --link docker/app.prod.ini $PHP_INI_DIR/conf.d/
RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

[tippexs]

Hi @p4block

I am using Unit with CakePHP and Symfony for more than 4 year and in my PHP Admin settings I am always setting some defaults

{
  "admin": {
    "memory_limit": "2G",
    "variables_order": "EGPCS",
    "expose_php": "0"
  },
  "user": {
    "display_errors": "0"
  }
}

The variable_order ar the most important thing as I make heavy use of the environment section in unit. Lately we have had an discussion about adding php.ini files dynamically. This can be done by PHP_INI_SCAN_DIR.

see: #969 (comment)

I agree we should start adding a section to our documentation where we start from the Language and not the application. Like Unit for PHP Developers should have a bunch of this information and some overview about why is Unit different from PHP-FPM and a setup with mod_php in Apache. Would that be helpful?

Cheers Timo

[p4block]

@tippexs

Although I like the unit-ful configuration and may switch to it later, right now I am simply using the default paths to configure PHP.

• /usr/local/etc/php/php.ini for global php, just copied from /usr/local/etc/php/php.ini-production
• /usr/local/etc/php/conf.d/app.ini for common settings related to my app.
• /usr/local/etc/php/conf.d/app.ini.prod for prod specific settings related to my app.

The default php.ini files (prod and dev one) shipped by php inside /usr/local/etc/php/ and recommended by them as best practices set "GPCS". So, under fpm with their upstream php config env vars work, and not under Unit. It's just that 99% of people using Unit never move the upstream php configs to their place, and now I wonder if it's even needed.

I'll also share my app specific settings for good measure, I hope this thread gives good value to future passerby.
app.ini

# Our settings
memory_limit = 2048M
upload_max_filesize = 2048M
post_max_size = 512M

# As per symfony's example dockerized app
expose_php = 0
session.use_strict_mode = 1
zend.detect_unicode = 0

; https://symfony.com/doc/current/performance.html
realpath_cache_size = 4096K
realpath_cache_ttl = 600
opcache.interned_strings_buffer = 16
opcache.max_accelerated_files = 20000
opcache.memory_consumption = 256

# Under Unit the container env vars don't get passed to the app without this
# It's possible to manually inject 1 by 1 in Unit's app config, but that's incredibly annoying
# EGPCS actually is the PHP default but it's revoked by their *container* php.ini files that we also copy
# As to why the problem only manifests in Unit, no clue
# https://github.com/nginx/unit/issues/302#issuecomment-1762387843
variables_order = EGPCS

app.ini.prod

opcache.preload_user = unit
opcache.preload = /www/config/preload.php
opcache.validate_timestamps = 0

PS: if you could share how you're running your entire setup I would be very thankful

[jeffdafoe]

I agree we should start adding a section to our documentation where we start from the Language and not the application. Like Unit for PHP Developers should have a bunch of this information and some overview about why is Unit different from PHP-FPM and a setup with mod_php in Apache. Would that be helpful?

Cheers Timo

That would definitely be a welcome addition.

Best, Jeff

[tippexs]

Moving this into a discussion to put all out new ideas and comments here! The topic about how Units PHP implementation differs from FPM or others is important to us and we will start working on better documentation and fixes.