diff options
| author | Andrew Clayton <a.clayton@nginx.com> | 2023-01-05 22:04:32 +0000 |
|---|---|---|
| committer | Andrew Clayton <a.clayton@nginx.com> | 2023-01-12 17:56:00 +0000 |
| commit | f3d05bba525c048d8ca905831a7729d7f90c94bc (patch) | |
| tree | 837654b4a32caac3764b08a44d59ce88c960ac69 | |
| parent | a560cbf992724eb0195544729d67286fd81f4bac (diff) | |
| download | unit-f3d05bba525c048d8ca905831a7729d7f90c94bc.tar.gz unit-f3d05bba525c048d8ca905831a7729d7f90c94bc.tar.bz2 | |
Python: Fix enabling of UTF-8 in some situations.
There was a couple of reports of Python applications failing due to the
following type of error
File "/opt/netbox/netbox/netbox/configuration.py", line 25, in _import
print(f"\U0001f9ec loaded config '{path}'")
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f9ec' in
position 0: ordinal not in range(128)
due to the use of Unicode text in the print() statement.
This only happened for python 3.8+ when using the "home" configuration
option as this meant we were going through the new PyConfig
configuration.
When using this new configuration method with the 'isolated' specific
API (for embedded Python) UTF-8 is disabled by default,
PyPreConfig->utf8_mode = 0.
To fix this we need to setup the Python pre config and enable utf-8
mode. However rather than enable utf-8 unconditionally we can set to it
to -1 so that it will use the LC_CTYPE environment variable to determine
whether to enable utf-8 mode or not. utf-8 mode will be enabled if
LC_CTYPE is either: C, POSIX or some specific UTF-8 locale. This is the
default utf8_mode setting when using the non-isolated PyPreConfig API.
Reported-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Tested-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Link: <https://peps.python.org/pep-0587/>
Link: <https://docs.python.org/3/c-api/init_config.html#c.PyPreConfig.utf8_mode>
Fixes: 491d0f70 ("Python: Added support for Python 3.11.")
Closes: <https://github.com/nginx/unit/issues/817>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
| -rw-r--r-- | src/python/nxt_python.c | 14 |
1 files changed, 14 insertions, 0 deletions
diff --git a/src/python/nxt_python.c b/src/python/nxt_python.c index 83f2544e..7c059649 100644 --- a/src/python/nxt_python.c +++ b/src/python/nxt_python.c @@ -78,9 +78,23 @@ nxt_python3_init_config(nxt_int_t pep405) PyConfig config; PyStatus status; nxt_int_t ret; + PyPreConfig preconfig; ret = NXT_ERROR; + PyPreConfig_InitIsolatedConfig(&preconfig); + /* + * Determine whether to use UTF-8 mode or not, UTF-8 + * will be enabled if LC_CTYPE is C, POSIX or some + * specific UTF-8 locale. + */ + preconfig.utf8_mode = -1; + + status = Py_PreInitialize(&preconfig); + if (PyStatus_Exception(status)) { + return ret; + } + PyConfig_InitIsolatedConfig(&config); if (pep405) { |