| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
@oopsoop2 on GitHub reported a performance issue related to the default
listen(2) backlog size of 511 on nginx. They found that increasing it
helped, nginx has a config option to configure this.
They would like to be able to do the same on Unit (which also defaults
to 511 on some systems). This seems reasonable.
NOTE: On Linux before commit 97c15fa38 ("socket: Use a default listen
backlog of -1 on Linux") we defaulted to 511. Since that commit we
default to the Kernels default, which before 5.4 is 128 and after is
4096.
This adds a new per-listener 'backlog' config option, e.g
{
"listeners": {
"[::1]:8080": {
"pass": "routes",
"backlog": 1024
},
}
...
}
This doesn't effect the control socket.
Closes: https://github.com/nginx/unit/issues/1384
Reported-by: <https://github.com/oopsoop2>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Unit generally creates an extra number of router threads (to handle
client connections, not incl the main thread) to match the number of
available CPUs.
There are cases when this can go wrong, e.g on a high CPU count machine
and Unit is being effectively limited to a few CPUs via the cgroups cpu
controller. So Unit may create a large number of router threads when
they are only going to effectively run on a couple of CPUs or so.
There may be other cases where you would like to tweak the number of
router threads, depending on your workload.
As it turns out it looks like it was intended to be made configurable
but was just never hooked up to the config system.
This adds a new '/settings/listen_threads' config option which can be
set like
{
"listen": {
...
},
"settings": {
"listen_threads": 2,
...
},
...
}
Before this patch (on a four cpu system)
$ ps -efL | grep router
andrew 419832 419829 419832 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 445145 0 5 03:31 pts/10 00:00:00 unit: router
andrew 419832 419829 445146 0 5 03:31 pts/10 00:00:00 unit: router
After, with a threads setting of 2
$ ps -efL | grep router
andrew 419832 419829 419832 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 3 Aug12 pts/10 00:00:00 unit: router
Closes: https://github.com/nginx/unit/issues/1042
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is a temporary support for chunked request bodies by converting
to Content-Length. This allows for processing of such requests until
a more permanent solution is developed.
A new configuration option "chunked_transform" has been added to enable
this feature. The option can be set as follows:
{
"settings": {
"chunked_transform": true
}
}
By default, this option is set to false, which retains the current
behaviour of rejecting chunked requests with a '411 Length Required'
status code.
Please note that this is an experimental implementation.
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
|
|
A common pattern was to declare variables in functions like
static nxt_str_t ...
Not sure why static, as they were being treated more like string
literals (and of course they are _not_ thread safe), let's actually make
them constants (qualifier wise).
This handles core code conversion.
Reviewed-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
In nxt_router_temp_conf() we have
rtcf = nxt_mp_zget(mp, sizeof(nxt_router_conf_t));
if (nxt_slow_path(rtcf == NULL)) {
goto fail;
}
If rtcf is NULL then we do
fail:
if (rtcf->tstr_state != NULL) {
nxt_tstr_state_release(rtcf->tstr_state);
}
In which case we will dereference the NULL pointer rtcf.
This patch re-works the goto labels to make them more specific to their
intended purpose and ensures we are freeing things which have been
allocated.
This was found by the clang static analyser.
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is the first commit in adding WebAssembly Component Model language
module support.
This just adds a new NXT_APP_WASM_WC type, required by subsequent
commits.
The WC stands for WASI_COMPONENT
This new module will have a type of 'wasm-wasi-component'.
Link: <https://github.com/nginx/unit/issues/1098>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Refactor.
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Cc: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Closes: <https://github.com/nginx/unit/pull/1062>
|
|
This is the first patch in adding WebAssembly language module support.
This just adds a new NXT_APP_WASM type, required by subsequent commits.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This makes it much more clear what's what.
This is in preparation for adding WebAssembly language module support.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Normally Unit responds to HTTP requests by including a header like
Server: Unit/1.30.0
however it can sometimes be beneficial to withhold the version
information and in this case just respond with
Server: Unit
This patch adds a new "settings.http" boolean option called
server_version, which defaults to true, in which case the full version
information is sent. However this can be set to false, e.g
"settings": {
"http": {
"server_version": false
}
},
in which case Unit responds without the version information as the
latter example above shows.
Link: <https://www.ietf.org/rfc/rfc9110.html#section-10.2.4>
Closes: <https://github.com/nginx/unit/issues/158>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
- Configuration: added "/config/settings/http/log_route".
Type: bool
Default: false
This adds configurability to the error log. It allows enabling and
disabling logs related to how the router performs selection of the
routes.
- HTTP: logging request line.
Log level: [notice]
The request line is essential to understand which logs correspond to
which request when reading the logs.
- HTTP: logging route that's been discarded.
Log level: [info]
- HTTP: logging route whose action is selected.
Log level: [notice]
- HTTP: logging when "fallback" action is taken.
Log level: [notice]
Closes: <https://github.com/nginx/unit/issues/758>
Link: <https://github.com/nginx/unit/pull/824>
Link: <https://github.com/nginx/unit/pull/839>
Suggested-by: Timo Stark <t.stark@nginx.com>
Suggested-by: Mark L Wood-Patrick <mwoodpatrick@gmail.com>
Suggested-by: Liam Crilly <liam@nginx.com>
Tested-by: Liam Crilly <liam@nginx.com>
Acked-by: Artem Konev <a.konev@f5.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
Currently when using Unix domain sockets for requests, if unit is
reconfigured then it will fail if it tries to bind(2) again to a Unix
domain socket with something like
2023/02/25 19:15:50 [alert] 35274#35274 bind(\"unix:/tmp/unit.sock\") failed (98: Address already in use)
When closing such a socket we really need to unlink(2) it. However that
presents a problem in that when running as root, while the main process
runs as root and creates the socket, it's the router process, that runs
as an unprivileged user, e.g nobody, that closes the socket and would
thus remove it, but couldn't due to not having permission, even if the
socket is mode 0666, you need write permissions on the containing
directory to remove a file.
There are several options to solve this, all with varying degrees of
complexity and utility.
1) Give the user who the router process runs as write permission to
the directory containing the listen sockets. These can then be
unlink(2)'d from the router process.
Simple and would work, but perhaps not the most elegant.
2) Using capabilities(7). The router process could temporarily attain
the CAP_DAC_OVERRIDE capability, unlink(7) the socket, then
relinquish the capability until required again.
These are Linux specific (other systems may have similar mechanisms
which would be extra work to support). There is also a, albeit
small, window where the router process is running with elevated
privileges.
3) Have the main process do the unlink(2), it is after all the process
that created the socket.
This is what this commit implements.
We create a new port IPC message type of NXT_PORT_MSG_SOCKET_UNLINK,
that is used by the router process to notify the main process about a
Unix domain socket to unlink(2).
Upon doing a reconfigure the router process will call
nxt_router_listen_socket_release() which will close the socket, we
extend this function in the case of non-abstract Unix domain sockets, so
that it will send a message to the main process containing a copy of the
nxt_sockaddr_t structure that will contain the filename of the socket.
In the main process the handler that we have defined,
nxt_main_port_socket_unlink_handler(), for this message type will run
and allow us to look for the socket in question in the listen_sockets
array and remove it and unlink(2) the socket.
This then allows the reconfigure to work if it tries to bind(2) again to
a socket that previously existed.
Link: <https://github.com/nginx/unit/issues/669>
Link: <https://github.com/nginx/unit/pull/735>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.
This buffer was always a little larger than needed due to allocating space
for the remote address _and_ port and the local address _and_ port. We
also allocate space for the local port separately.
->{local,remote}->length includes the port number and ':' and also the
'[]' for IPv6. E.g [2001:db8::1]:8080
->{local,remote}->address_length represents the length of the unadorned
IP address. E.g 2001:db8::1
Update the buffer size so that we only allocate what is actually needed.
Suggested-by: Zhidao HONG <z.hong@f5.com>
Cc: Zhidao HONG <z.hong@f5.com>
Reviewed-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This fixes an issue reported by @Peter2121 on GitHub.
In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.
The req structure comprises various members with the final member being
an array (specified as a flexible array member, with its actual length
denoted by the req->fields_count member) of nxt_unit_field_t's. These
structures specify the length and offset for the various request headers
name/value pairs which are stored after some request metadata that is
stored immediately after this array of structs as individual nul
terminated strings.
After this we have the body content data (if any). So it looks a little
like
(gdb) x /64bs 0x7f38c976e060
0x7f38c976e060: "\353\346\244\t\006" <-- First nxt_unit_field_t
0x7f38c976e066: ""
0x7f38c976e067: ""
0x7f38c976e068: "T\001"
0x7f38c976e06b: ""
0x7f38c976e06c: "Z\001"
0x7f38c976e06f: ""
...
0x7f38c976e170: "\362#\244\v$" <-- Last nxt_unit_field_t
0x7f38c976e176: ""
0x7f38c976e177: ""
0x7f38c976e178: "\342\002"
0x7f38c976e17b: ""
0x7f38c976e17c: "\352\002"
0x7f38c976e17f: ""
0x7f38c976e180: "POST" <-- Start of request metadata
0x7f38c976e185: "HTTP/1.1"
0x7f38c976e18e: "unix:"
0x7f38c976e194: "unix:/dev/shm/842.sock"
0x7f38c976e1ab: ""
0x7f38c976e1ac: "fedora"
0x7f38c976e1b3: "/842.php"
0x7f38c976e1bc: "HTTP_HOST" <-- Start of header fields
0x7f38c976e1c6: "fedora"
0x7f38c976e1cd: "HTTP_X_FORWARDED_PROTO"
0x7f38c976e1e4: "https"
...
0x7f38c976e45a: "HTTP_COOKIE"
0x7f38c976e466: "PHPSESSID=8apkg25r9s9vju3pi085i21eh4"
0x7f38c976e48b: "public_form=sended" <-- Body content
Well that's how things are supposed to look! When using Unix domain
sockets what we actually got looked like
...
0x7f6141f3445a: "HTTP_COOKIE"
0x7f6141f34466: "PHPSESSID=uo5b2nu9buijkc89jotbgmd60vpublic_form=sended"
Here, the body content (from a POST for example) has been appended
straight onto the end of the last header field value. In this case
corrupting the PHP session cookie. The body content would still be
found by the application as its offset into this buffer is correct.
This problem was actually caused by a0327445 ("PHP: allowed to specify
URLs without a trailing '/'.") which added an extra item into this
request buffer specifying the port number that unit is listening on that
handled this request.
Unfortunately when I wrote that patch I didn't increase the size of this
request buffer to accommodate it.
When using normal TCP sockets we actually end up allocating more space
than required for this buffer, we track the end of this buffer up to
where the body content would go and so we have a few spare bytes between
the nul byte of the last field header value and the start of the body
content.
When using Unix domain sockets, they have no associated port number and
thus the port number has a length of 0 bytes, but we still write a '\0'
in there using up a byte that we didn't account for, this causes us to
loose the nul byte of the last header fields value causing the body data
to be appended to the last header field value.
The fix is simple, account for the local port length, we also add 1 to
it, this covers the nul byte, even if there is no port as with Unix
domain sockets.
Closes: <https://github.com/nginx/unit/issues/842>
Fixes: a0327445 ("PHP: allowed to specify URLs without a trailing '/'.")
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This commit fixed the njs memory leak happened in the config validation, updating and http requests.
|
|
|
|
|
|
It's for the introduction of njs support.
For each option that supports native variable and JS template literals introduced next,
it's unified as template string.
No functional changes.
|
|
Both @lucatacconi & @mwoodpatrick reported what appears to be the same
issue on GitHub. Namely that when using the PHP language module and
trying to access a URL that is a directory but without specifying the
trailing '/', they were getting a '503 Service Unavailable' error.
Note: This is when _not_ using the 'script' option.
E.g with the following config
{
"listeners": {
"[::1]:8080": {
"pass": "applications/php"
}
},
"applications": {
"php": {
"type": "php",
"root": "/var/tmp/unit-php"
}
}
}
and with a directory path of /var/tmp/unit-php/foo containing an
index.php, you would see the following
$ curl http://localhost/foo
<title>Error 503</title>
Error 503
However
$ curl http://localhost/foo/
would work and serve up the index.php
This commit fixes the above so you get the desired behaviour without
specifying the trailing '/' by doing the following
1] If the URL doesn't end in .php and doesn't have a trailing '/'
then check if the requested path is a directory.
2) If it is a directory then create a 301 re-direct pointing to it.
This matches the behaviour of the likes of nginx, Apache and
lighttpd.
This also matches the behaviour of the "share" action in Unit.
This doesn't effect the behaviour of the 'script' option which bypasses
the nxt_php_dynamic_request() function.
This also adds a couple of tests to test/test_php_application.py to
ensure this continues to work.
Closes: <https://github.com/nginx/unit/issues/717>
Closes: <https://github.com/nginx/unit/issues/753>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is a preparatory patch that renames the 'local' and 'local_length'
members of the nxt_unit_request_t structure to 'local_addr' and
'local_addr_length' in preparation for the adding of 'local_port' and
'local_port_length' members.
Suggested-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
|
|
No functional changes.
|
|
This commit adds the variables $arg_NAME, $header_NAME, and $cookie_NAME.
|
|
|
|
This makes the replacement of forwarded request header
like client_ip and protocol more generic.
It's a prerequirement for protocol replacement.
No functional changes.
|
|
No functional changes.
|
|
The message tracking is unused since 1d84b9e4b459 commit.
This fixes the issue found by Coverity (CID 376263).
|
|
No functional changes.
|
|
The previous commit added more generic APIs for handling
NXT_CONF_VALUE_ARRAY and non-NXT_CONF_VALUE_ARRAY together.
Modify calling code to remove special cases for arrays and
non-arrays, taking special care that the path for non arrays is
logically equivalent to the previous special cased code.
Use the now-generic array code only.
|
|
The reference to the access_log structure is stored in the current
nxt_router_conf_t and the global nxt_router_t. When the reference is copied,
the reference counter should be adjusted accordingly.
This closes #593 issue on GitHub.
|
|
Application process started with shared port (and queue) already configured.
But still waits for PORT_ACK message from router to start request processing
(so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may
produce response and send it to router before the router have the information
about the application process. This is a subject of further optimizations.
|
|
|
|
|
|
Introducting application graceful stop. For now only used when application
process reach request limit value.
This closes #585 issue on GitHub.
|
|
port->app field is not thread safe and should be used in main thread only.
To release port after request processing, application reference should be
obtained from corresponding request descriptor.
|
|
|
|
No functional changes.
|
|
This commit introduces the replacement of the client address based on the value
of a specified HTTP header. This is intended for use when Unit is placed
behind a reverse proxy like nginx or a CDN.
You must specify the source addresses of the trusted proxies. This can be
accomplished with any valid IP pattern supported by Unit's match block:
["10.0.0.1", "10.4.0.0/16", "!192.168.1.1"]
The feature is configured per listener.
The client address replacement functionality only operates when there is a
source IP match and the specified header is present. Typically this would be
an 'X-Forwarded-For' header.
{
"listeners": {
"127.0.0.1:8080": {
"client_ip": {
"header": "X-Forwarded-For",
"source": [
"10.0.0.0/8"
]
},
"pass": "applications/my_app"
},
}
}
If a request occurs and Unit receives a header like below:
"X-Forwarded-For: 84.123.23.23"
By default, Unit trusts the last rightmost IP in the header, so REMOTE_ADDR
will be set to 84.123.23.23 if the connection originated from 10.0.0.0/8.
If Unit runs behind consecutive reverse proxies and receives a header similar
to the following:
"X-Forwarded-For: 84.123.23.23, 10.0.0.254"
You will need to enable "recursive" checking, which walks the header from
last address to first and chooses the first non-trusted address it finds.
{
"listeners": {
"127.0.0.1:8080": {
"client_ip": {
"header": "X-Forwarded-For",
"source": [
"10.0.0.0/8"
]
"recursive": true,
},
"pass": "applications/my_app"
},
}
}
If a connection from 10.0.0.0/8 occurs, the chain is walked. Here, 10.0.0.254
is also a trusted address so the client address will be replaced with
84.123.23.23.
If all IP addresses in the header are trusted, the client address is set to
the first address in the header:
If 10.0.0.0/8 is trusted and "X-Forwarded-For: 10.0.0.3, 10.0.0.2, 10.0.0.1",
the client address will be replaced with 10.0.0.3.
|
|
When processing a restart request, the router sends a QUIT message to all
existing processes of the application. Then, a new shared application port is
created to ensure that new requests won't be handled by the old processes of
the application.
|
|
No functional changes.
|
|
No functional changes.
|
|
To support TLS sessions, Unit uses the OpenSSL built-in session cache; the
cache_size option defines the number sessions to store. To disable the feather,
the option must be zero.
|
|
A new application thread port message can be processed in the router after the
application is removed from the router. Assertion for this case is replaced by
a condition to store the new thread port until receiving the stop notification
from the application process.
|
|
Because of the incorrect 'last' field assignment, multiple listeners with
a TLS certificate did not initialize properly, which caused a router crash
while establishing a connection.
Test with multiple TLS listeners added.
The issue was introduced in the c548e46fe516 commit.
This closes #561 issue on GitHub.
|
|
To perform various configuration operations on SSL_CTX, OpenSSL provides
SSL_CONF_cmd(). Specifically, to configure ciphers for a listener,
"CipherString" and "Ciphersuites" file commands are used:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html
This feature can be configured in the "tls/conf_commands" section.
|
|
A crash was caused by an incorrect timer handler nxt_h1p_idle_timeout() if
SSL_shutdown() returned SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE.
The flag SSL_RECEIVED_SHUTDOWN is used to avoid getting SSL_ERROR_WANT_READ, so
the server won't wait for a close notification from a client.
For SSL_ERROR_WANT_WRITE, a correct timer handler is set up.
|
|
This patch fixes a possible race between the nxt_router_conf_wait() and
nxt_router_listen_socket_release() function calls and improves the 7f1b2eaa2d58
commit fix.
|
