| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This is a preparatory patch that factors out the asyncio event loop
creation code from nxt_python_asgi_ctx_data_alloc() into its own
function, to facilitate being called multiple times.
This a part of the work to move away from using the
asyncio.get_event_loop() function due to it no longer creating event
loops if there wasn't one running.
See the following commit for the gory details.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This allows one to simply run `./configure` and expect it to
produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install`
would simply fail, recommending to set `--prefix` or `DESTDIR`,
but that recommendation was incomplete at best, since it didn't
set many of the subdirs needed for a good organization.
Setting `DESTDIR` was even worse, since that shouldn't even affect
an installation (it is required to be transparent to the
installation).
/usr/local is the historic Unix standard path to use for
installations from source made manually by the admin of the
system. Some package managers (Homebrew, I'm looking specifically
at you) have abused that path to install their things, but 1) it's
not our fault that someone else incorrectly abuses that path (and
they seem to be fixing it for newer archs; e.g., they started
using /opt/homebrew for Apple Silicon), 2) there's no better path
than /usr/local, 3) we still allow changing it for systems where
this might not be the desired path (MacOS Intel with hombrew), and
4) it's _the standard_.
See a related conversation with Ingo (OpenBSD maintainer):
On 7/27/22 16:16, Ingo Schwarze wrote:
> Hi Alejandro,
[...]
>
> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 07:07:18PM +0200:
>> On 7/24/22 16:57, Ingo Schwarze wrote:
>>> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 01:20:46PM +0200:
>
>>>> /usr/local is for sysadmins to build from source;
>
>>> Doing that is *very* strongly discouraged on OpenBSD.
>
>> I guess that's why the directory was reused in the BSDs to install ports
>> (probably ports were installed by the sysadmin there, and by extension,
>> ports are now always installed there, but that's just a guess).
>
> Maybe. In any case, the practice of using /usr/local for packages
> created from ports is significantly older than the recommendation
> to refrain from using upstream "make install" outside the ports
> framework.
>
> * The FreeBSD ports framework was started by Jordan Hubbard in 1993.
> * The ports framework was ported from FreeBSD to OpenBSD
> by Niklas Hallqvist in 1996.
> * NetBSD pkgsrc was forked from FreeBSD ports by Alistair G. Crooks
> and Hubert Feyrer in 1997.
>
> I failed to quickly find Jordan's original version, but rev. 1.1
> of /usr/ports/infrastructure/mk/bsd.port.mk in OpenBSD (dated Jun 3
> 22:47:10 1996 UTC) already said
>
> LOCALBASE ?= /usr/local
> PREFIX ?= ${LOCALBASE}
>
[...]
>> I had a discussion in NGINX Unit about it, and
>> the decission for now has been: "support prefix=/usr/local for default
>> manual installation through the Makefile, and let BSD users adjust to
>> their preferred path".
>
> That's an *excellent* solution for the task, thanks for doing it
> the right way. By setting PREFIX=/usr/local by default in the
> upstream Makefile, you are minimizing the work for *BSD porters.
>
> The BSD ports frameworks will typically run the upstreak "make install"
> with the variable DESTDIR set to a custom value, for example
>
> DESTDIR=/usr/ports/pobj/groff-1.23.0/fake-amd64
>
> so if the upstream Makefile sets PREFIX=/usr/local ,
> that's perfect, everything gets installed to the right place
> without an intervention by the person doing the porting.
>
> Of course, if the upstream Makefile would use some other PREFIX,
> that would not be a huge obstacle. All we have to do in that case
> is pass the option --prefix=/usr/local to the ./configure script,
> or something equivalent if the software isn't using GNU configure.
>
>> We were concerned that we might get collisions
>> with the BSD port also installing in /usr/local, but that's the least
>> evil (and considering BSD users don't typically run `make install`, it's
>> not so bad).
>
> It's not bad at all. It's perfect.
>
> Of course, if a user wants to install *without* the ports framework,
> they have to provide their own --prefix. But that's not an issue
> because it is easy to do, and installing without a port is discouraged
> anyway.
===
Directory variables should never contain a trailing slash (I've
learned that the hard way, where some things would break
unexpectedly). Especially, make(1) is likely to have problems
when things have double slashes or a trailing slash, since it
treats filenames as text strings. I've removed the trailing slash
from the prefix, and added it to the derivate variables just after
the prefix. pkg-config(1) also expects directory variables to have
no trailing slash.
===
I also removed the code that would set variables as depending on
the prefix if they didn't start with a slash, because that is a
rather non-obvious behavior, and things should not always depend
on prefix, but other dirs such as $(runstatedir), so if we keep
a similar behavior it would be very unreliable. Better keep
variables intact if set, or use the default if unset.
===
Print the real defaults for ./configure --help, rather than the actual
values.
===
I used a subdirectory under the standard /var/lib for NXT_STATE,
instead of a homemade "state" dir that does the same thing.
===
Modified the Makefile to create some dirs that weren't being
created, and also remove those that weren't being removed in
uninstall, probably because someone forgot to add them.
===
Add new options for setting the new variables, and rename some to be
consistent with the standard names. Keep the old ones at configuration
time for compatibility, but mark them as deprecated. Don't keep the old
ones at exec time.
===
A summary of the default config is:
Unit configuration summary:
bin directory: ............. "/usr/local/bin"
sbin directory: ............ "/usr/local/sbin"
lib directory: ............. "/usr/local/lib"
include directory: ......... "/usr/local/include"
man pages directory: ....... "/usr/local/share/man"
modules directory: ......... "/usr/local/lib/unit/modules"
state directory: ........... "/usr/local/var/lib/unit"
tmp directory: ............. "/tmp"
pid file: .................. "/usr/local/var/run/unit/unit.pid"
log file: .................. "/usr/local/var/log/unit/unit.log"
control API socket: ........ "unix:/usr/local/var/run/unit/control.unit.sock"
Link: <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html>
Link: <https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html>
Reviewed-by: Artem Konev <a.konev@f5.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Tested-by: Andrew Clayton <a.clayton@nginx.com>
Reviewed-by: Konstantin Pavlov <thresh@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
Since the previous commit, we now properly handle 403 Forbidden & 404
Not Found errors in the PHP language module.
This adds a test for 403 Forbidden to test/test_php_application.py, but
also fixes a test in test/test_php_targets.py where we were checking for
503 but should have been a 404, which we now do.
Acked-by: Alejandro Colomar <alx@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
[ Incorporates a couple of small test cleanups from Andrei ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Previously the PHP module would produce one of four status codes
200 OK
301 Moved Permanently
500 Internal Server Error
503 Service Unavailable
200 for successful requests, 301 for cases where the url was a directory
without a trailing '/', 500 for bad PHP or non-existing PHP file and 503
for all other errors.
With this commit we now handle missing files and directories, returning
404 Not Found and files and directories that don't allow access,
returning 403 Forbidden.
We do these checks in two places, when we check if we should do a
directory redirect (bar -> bar/) and in the nxt_php_execute() function.
One snag with the latter is that the php_execute_script() function only
returns success/failure (no reason). However while it took a
zend_file_handle structure with the filename of the script to run, we
can instead pass through an already opened file-pointer (FILE *) via
that structure. So we can try opening the script ourselves and do the
required checks before calling php_execute_script().
We also make use of the zend_stream_init_fp() function that initialises
the zend_file_handle structure if it's available otherwise we use our
own version. This is good because the zend_file_handle structure has
changed over time and the zend_stream_init_fp() function should change
with it.
Closes: <https://github.com/nginx/unit/issues/767>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Create a const char *filename variable to hold
ctx->script_filename.start, which is a much more manageable name and
will negate the need for any more casting in the following commit when
we switch to using a FILE * instead of a filename in
php_execute_script().
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Where possible make use of the zend_stream_init_filename() function
introduced in PHP 7.4.
This is essentially a preparatory patch for switching to using an
already opened file-pointer in nxt_php_execute(). While wrapping this
new code in a PHP version check with a fallback to our own function is
perhaps slightly overkill, it does reduce the diff of the commit that
switches to a FILE *.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
We're going to use zend_stream_init_filename in a following commit. To
reduce the diff of that change, move the current code that will be
replaced, to a function that has the same interface.
We use strlen(3) here to be able to use an interface without passing the
length, but we will remove that call in a following code, so it has no
performance issues.
Co-developed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
|
|
|
|
|
|
This commit is to loop through the request objects headers,
arguments, and cookies.
|
|
@dward on GitHub reported an issue with a URL like
http://foo.bar/test.php?blah=test.php/foo
where we would end up trying to run the script
test.php?blah=test.php
In the PHP module the format 'file.php/' is treated as a special case in
nxt_php_dynamic_request() where we check the _path_ part of the url for
the string '.php/'.
The problem is that the path actually also contains the query string,
thus we were finding 'test.php/' in the above URL and treating that
whole path as the script to run.
The fix is simple, replace the strstr(3) with a memmem(3), where we can
limit the amount of path we use for the check.
The trick here and what is not obvious from the code is that while
path.start points to the whole path including the query string,
path.length only contains the length of the _path_ part.
NOTE: memmem(3) is a GNU extension and is neither specified by POSIX or
ISO C, however it is available on a number of other systems, including:
FreeBSD, OpenBSD, NetBSD, illumos, and macOS.
If it comes to it we can implement a simple alternative for systems
which lack memmem(3).
This also adds a test case (provided by @dward) to cover this.
Closes: <https://github.com/nginx/unit/issues/781>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> [test]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The nxt_websocket_header_t structure defines the layout of a websocket
frame header. As the websocket frame is mapped directly onto this
structure its layout needs to match how it's coming off the network.
The network being big endian means on big endian systems the structure
layout can simply match that of the websocket frame header. On little
endian systems we need to reverse the two bytes.
This was done via the BYTE_ORDER, BIG_ENDIAN and LITTLE_ENDIAN macros,
however these are not universal, e.g they are _not_ defined on illumos
(OpenSolaris / OpenIndiana) and so we get the following compiler errors
In file included from src/nxt_h1proto.c:12:0:
src/nxt_websocket_header.h:25:13: error: duplicate member 'opcode'
uint8_t opcode:4;
^~~~~~
src/nxt_websocket_header.h:26:13: error: duplicate member 'rsv3'
uint8_t rsv3:1;
^~~~
src/nxt_websocket_header.h:27:13: error: duplicate member 'rsv2'
uint8_t rsv2:1;
^~~~
src/nxt_websocket_header.h:28:13: error: duplicate member 'rsv1'
uint8_t rsv1:1;
^~~~
src/nxt_websocket_header.h:29:13: error: duplicate member 'fin'
uint8_t fin:1;
^~~
src/nxt_websocket_header.h:31:13: error: duplicate member 'payload_len'
uint8_t payload_len:7;
^~~~~~~~~~~
src/nxt_websocket_header.h:32:13: error: duplicate member 'mask'
uint8_t mask:1;
^~~~
This commit fixes that by using the new NXT_HAVE_{BIG,LITTLE}_ENDIAN
macros introduced in the previous commit.
Closes: <https://github.com/nginx/unit/issues/297>
Fixes: e501c74 ("Introducing websocket support in router and libunit.")
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
In configure we set NXT_HAVE_LITTLE_ENDIAN for i386, amd64 and x86_64.
However that misses at least AArch64 (arm64) where it's usually run in
little endian mode.
However none of that really matters as NXT_HAVE_LITTLE_ENDIAN isn't used
anywhere. So why this patch?
The only place we need to explicitly know about endianness is the
nxt_websocket_header_t structure where we lay it out differently
depending on endianness.
This is currently done using BYTE_ORDER, LITTLE_ENDIAN and BIG_ENDIAN
macros.
However on at least illumos (OpenSolaris / OpenIndiana) those macros are
not defined and we get compiler errors due to duplicate structure
members.
So let's use our own NXT_HAVE_{BIG,LITTLE}_ENDIAN macros. However it
would be better to detect endianness programmatically as some
architectures can run in either mode, e.g Linux used to run in big
endian on PowerPC but has since switched to little endian (to match
x86).
This commit adds an auto/endian script (using a slightly modified
version of the test program from nginx's auto script), that checks for
the endianness of the platform being built on. E.g
checking for endianness ... little endian
The next commit will switch the nxt_websocket_header_t structure over to
these new macros.
Link: <https://github.com/nginx/unit/pull/298>
Link: <https://developer.ibm.com/articles/l-power-little-endian-faq-trs/>
Tested-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
There was a couple of reports of Python applications failing due to the
following type of error
File "/opt/netbox/netbox/netbox/configuration.py", line 25, in _import
print(f"\U0001f9ec loaded config '{path}'")
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f9ec' in
position 0: ordinal not in range(128)
due to the use of Unicode text in the print() statement.
This only happened for python 3.8+ when using the "home" configuration
option as this meant we were going through the new PyConfig
configuration.
When using this new configuration method with the 'isolated' specific
API (for embedded Python) UTF-8 is disabled by default,
PyPreConfig->utf8_mode = 0.
To fix this we need to setup the Python pre config and enable utf-8
mode. However rather than enable utf-8 unconditionally we can set to it
to -1 so that it will use the LC_CTYPE environment variable to determine
whether to enable utf-8 mode or not. utf-8 mode will be enabled if
LC_CTYPE is either: C, POSIX or some specific UTF-8 locale. This is the
default utf8_mode setting when using the non-isolated PyPreConfig API.
Reported-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Tested-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Link: <https://peps.python.org/pep-0587/>
Link: <https://docs.python.org/3/c-api/init_config.html#c.PyPreConfig.utf8_mode>
Fixes: 491d0f70 ("Python: Added support for Python 3.11.")
Closes: <https://github.com/nginx/unit/issues/817>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is a preparatory patch for future work and cleans up the code a
little in the Python 3.8+ variant of nxt_python3_init_config().
The main advantage being we no longer have calls to PyConfig_Clear() in
two different paths.
The variables have a little extra space in their declarations to allow
for the next patch which introduces a variable with a longer type name,
which will help reduce the size of the diff.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Reported-by: Liam Crilly <liam@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
mktemp(1) in macOS uses a weird directory where only the running user
has permissions. If we use that for the welcome website, unitd(8) won't
be able to read the page. Use a directory at $HOME before trying a tmpdir.
Reported-by: Liam Crilly <lcrilly@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
Reported-by: Liam Crilly <lcrilly@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
These directories are used in the Makefile to determine status of a target.
|
|
While at it, fixed changelogs generation for Python 3.10 as well.
|
|
|
|
'sudo' was misplaced.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
This prevents accidents, which are likely to happen especially with quotes.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
pkg-config package is named differently on supported rpm-based systems:
- Amazon Linux 2 has pkgconfig
- Fedora has pkgconf-pkg-config
- RHEL 7 has pkgconfig
- RHEL 8 and 9 have pkgconfig-pkg-config
What they share in common is they all provide 'pkgconfig', which we can
use in the spec file so we don't have to specify it per-OS.
|
|
|
|
|
|
|
|
This script combines the old setup-unit (as the repo-config command),
with new functionality, to provide an easy welcome website for
first-time users, and also some more commands that are useful for
administrating a running unitd(8) instance.
Suggested-by: Liam Crilly <liam@nginx.com>
Cc: Konstantin Pavlov <thresh@nginx.com>
Cc: Artem Konev <a.konev@f5.com>
Cc: Timo Start <t.stark@nginx.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
|
|
This patch gives users the option to set a `"prefix"` attribute
for Python applications, either at the top level or for specific
`"target"`s. If the attribute is present, the value of `"prefix"`
must be a string beginning with `"/"`. If the value of the `"prefix"`
attribute is longer than 1 character and ends in `"/"`, the
trailing `"/"` is stripped.
The purpose of the `"prefix"` attribute is to set the `SCRIPT_NAME`
context value for WSGI applications and the `root_path` context
value for ASGI applications, allowing applications to properly route
requests regardless of the path that the server uses to expose the
application.
The context value is only set if the request's URL path begins with
the value of the `"prefix"` attribute. In all other cases, the
`SCRIPT_NAME` or `root_path` values are not set. In addition, for
WSGI applications, the value of `"prefix"` will be stripped from
the beginning of the request's URL path before it is sent to the
application.
Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Artem Konev <artem.konev@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
|
|
Fixes https://github.com/nginx/unit/issues/728
Refs https://github.com/nginx/unit/issues/718
|
|
|
|
|
|
Added tests for the "large_header_buffer_size" and
"large_header_buffers" configuration options.
|
|
This is an extension to the previous commit, which made
large_header_buffer_size a valid configuration setting.
This commit makes a related value, large_header_buffers, a valid
configuration setting.
While large_header_buffer_size effectively limits the maximum size of
any single header (although unit will try to pack multiple headers into
a buffer if they wholly fit).
large_header_buffers limits how many of these 'large' buffers are
available. It makes sense to also allow this to be user set.
large_header_buffers is already set by the configuration system in
nxt_router.c it just isn't set as a valid config option in
nxt_conf_validation.c
With this change users can set this option in their config if required
by the following
"settings": {
"http": {
"large_header_buffers": 8
}
},
It retains its default value of 4 if this is not set.
NOTE: This is being released as undocumented and subject to change as it
exposes internal workings of unit.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
@JanMikes and @tagur87 on GitHub both reported issues with long URLs
that were exceeding the 8192 byte large_header_buffer_size setting,
which resulted in a HTTP 431 error (Request Header Fields Too Large).
This can be resolved in the code by updating the following line in
src/nxt_router.c::nxt_router_conf_create()
skcf->large_header_buffer_size = 8192;
However, requiring users to modify unit and install custom versions is
less than ideal. We could increase the value, but to what?
This commit takes the option of allowing the user to set this option in
their config by making large_header_buffer_size a valid configuration
setting.
large_header_buffer_size is already set by the configuration system in
nxt_router.c it just isn't set as a valid config option in
nxt_conf_validation.c
With this change users can set this option in their config if required
by the following
"settings": {
"http": {
"large_header_buffer_size": 16384
}
},
It retains its default value of 8192 bytes if this is not set.
With this commit, without the above setting or too low a value, with a
long URL you get a 431 error. With the above setting set to a large
enough value, the request is successful.
NOTE: This setting really determines the maximum size of any single
header _value_. Also, unit will try and place multiple values
into a buffer _if_ they fully fit.
NOTE: This is being released as undocumented and subject to change as it
exposes internal workings of unit.
Closes: <https://github.com/nginx/unit/issues/521>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Hide expected alerts by default.
Silence succesfull "go build" information.
|
