| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Initially, variable query was designed to accomodate both synchronous
and asynchronous operations. However, upon consideration of actual
requirements, we recognized that asynchronous support was not needed.
The refactoring ensures that the success or failure of the variable
query operation is now directly indicated by its return value. This
change streamlines the function's usage and enhances code clarity,
as it facilitates immediate error handling without the need for
asynchronous callbacks or additional error checking functions.
Note the patch only works for Unit native variables but not njs
variables.
|
|
@oopsoop2 on GitHub reported a performance issue related to the default
listen(2) backlog size of 511 on nginx. They found that increasing it
helped, nginx has a config option to configure this.
They would like to be able to do the same on Unit (which also defaults
to 511 on some systems). This seems reasonable.
NOTE: On Linux before commit 97c15fa38 ("socket: Use a default listen
backlog of -1 on Linux") we defaulted to 511. Since that commit we
default to the Kernels default, which before 5.4 is 128 and after is
4096.
This adds a new per-listener 'backlog' config option, e.g
{
"listeners": {
"[::1]:8080": {
"pass": "routes",
"backlog": 1024
},
}
...
}
This doesn't effect the control socket.
Closes: https://github.com/nginx/unit/issues/1384
Reported-by: <https://github.com/oopsoop2>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
On FreeBSD, OpenBSD & macOS we use a default listen(2) backlog of -1
which means use the OS's default value.
On Linux (and others) we used a hard coded value of 511, presumably due
to this comment
/* Linux, Solaris, and NetBSD treat negative value as 0. */
On Linux (at least since 2.4), this is wrong, Linux treats -1 (and so
on) as use the OS's default (net.core.somaxconn). See this code in
net/socket.c::__sys_listen()
if ((unsigned int)backlog > somaxconn)
backlog = somaxconn;
On Linux prior to 5.4 somaxconn defaulted to 128, since 5.4 it defaults
to 4096.
We've had complaints that a listen backlog of 511 is too small. This
would help in those cases.
Unless they are on an old Kernel, in which case it's worse, but then the
plan is to also make this configurable. This would effect RHEL 8, which
is based on 4.10, however they seem to set somaxconn to 2048, so that's
fine.
Another advantage of using -1 is that we will automatically keep up to
date with the kernels default value.
Before this change
$ ss -tunxlp | grep unit
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
u_str LISTEN 0 511 /opt/unit/control.unit.sock.tmp 4302333 * 0 users:(("unitd",pid=18290,fd=6),("unitd",pid=18289,fd=6),("unitd",pid=18287,fd=6))
tcp LISTEN 0 511 127.0.0.1:8080 0.0.0.0:* users:(("unitd",pid=18290,fd=12))
tcp LISTEN 0 511 [::1]:8080 [::]:* users:(("unitd",pid=18290,fd=11))
After
$ ss -tunxlp | grep unit
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
u_str LISTEN 0 4096 /opt/unit/control.unit.sock.tmp 5408464 * 0 users:(("unitd",pid=132442,fd=6),("unitd",pid=132441,fd=6),("unitd",pid=132439,fd=6))
tcp LISTEN 0 4096 127.0.0.1:8080 0.0.0.0:* users:(("unitd",pid=132442,fd=12))
tcp LISTEN 0 4096 [::1]:8080 [::]:* users:(("unitd",pid=132442,fd=11))
Link: <https://github.com/nginx/unit/issues/1384>
Link: <https://lore.kernel.org/netdev/20191030163620.140387-1-edumazet@google.com/>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Unit generally creates an extra number of router threads (to handle
client connections, not incl the main thread) to match the number of
available CPUs.
There are cases when this can go wrong, e.g on a high CPU count machine
and Unit is being effectively limited to a few CPUs via the cgroups cpu
controller. So Unit may create a large number of router threads when
they are only going to effectively run on a couple of CPUs or so.
There may be other cases where you would like to tweak the number of
router threads, depending on your workload.
As it turns out it looks like it was intended to be made configurable
but was just never hooked up to the config system.
This adds a new '/settings/listen_threads' config option which can be
set like
{
"listen": {
...
},
"settings": {
"listen_threads": 2,
...
},
...
}
Before this patch (on a four cpu system)
$ ps -efL | grep router
andrew 419832 419829 419832 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 445145 0 5 03:31 pts/10 00:00:00 unit: router
andrew 419832 419829 445146 0 5 03:31 pts/10 00:00:00 unit: router
After, with a threads setting of 2
$ ps -efL | grep router
andrew 419832 419829 419832 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 3 Aug12 pts/10 00:00:00 unit: router
Closes: https://github.com/nginx/unit/issues/1042
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
At startup, the unit router process creates a number of threads, it
tries to create the same number of threads (not incl the main thread) as
there are 'cpus' in the system.
On Linux the number of available cpus is determined via a call to
sysconf(_SC_NPROCESSORS_ONLN);
in a lot of cases this produces the right result, i.e. on a four cpu
system this will return 4.
However this can break down if unit has been restricted in the cpus it's
allowed to run on via something like cpuset()'s and/or
sched_setaffinity(2).
For example, on a four 'cpu' system, starting unit will create an extra
4 router threads
$ /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 234102 234099 234102 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234103 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234104 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234105 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234106 0 5 17:00 pts/10 00:00:00 unit: router
Say we want to limit unit to two cpus, i.e.
$ taskset -a -c 2-3 /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 235772 235769 235772 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235773 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235774 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235775 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235776 0 5 17:08 pts/10 00:00:00 unit: router
So despite limiting unit to two cpus
$ grep Cpus_allowed_list /proc/235772/status
Cpus_allowed_list: 2-3
It still created 4 threads, probably not such an issue in this case, but
if we had a 64 'cpu' system and wanted to limit unit two cpus, then we'd
have 64 threads vying to run on two cpus and with our spinlock
implementation this can cause a lot of thread scheduling and congestion
overhead.
Besides, our intention is currently to create nr router threads == nr
cpus.
To resolve this, on Linux at least, this patch makes use of
sched_getaffinity(2) to determine what cpus unit is actually allowed to
run on.
We still use the result of
sysconf(_SC_NPROCESSORS_ONLN);
as a fallback, we also use its result to allocate the required cpuset
size (where sched_getaffinity() will store its result) as the standard
cpu_set_t only has space to store 1023 cpus.
So with this patch if we try to limit unit to two cpus we now get
$ taskset -a -c 2-3 /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 236887 236884 236887 0 3 17:20 pts/10 00:00:00 unit: router
andrew 236887 236884 236888 0 3 17:20 pts/10 00:00:00 unit: router
andrew 236887 236884 236889 0 3 17:20 pts/10 00:00:00 unit: router
This also applies to the likes of docker, if you run docker with the
--cpuset-cpus="" option, unit will now create a number of router threads
that matches the cpu count specified.
Perhaps useful if you are running a number of unit docker instances on a
high cpu count machine.
Link: <https://github.com/nginx/unit/issues/1042>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This will help to better determine the number of router threads to
create in certain situations.
Unlike sysconf(_SC_NPROCESSORS_ONLN) this takes into account per-process
cpu allowed masks as set by sched_setaffinity(2)/cpusets etc.
So while a system may have 64 on-line cpu's, Unit itself may be limited
to using just four of them in which case we should create four extra
router threads, not sixty-four!
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Unit now shows the loaded language modules under /status/modules. This
functionality was added in commit 707f4ef82 ("status: Show list of
loaded language modules").
[ Commit message - Andrew ]
Co-developed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This was renamed to be more accurate as /status is not just about
statistics, as it also now shows the loaded language modules.
[ Commit message - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Seems the README was never updated to account for WebAssembly...
Reported-by: Palmese Davide Mattia <https://github.com/PalmeseMattia>
Closes: https://github.com/nginx/unit/issues/1376
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
It's possible to have two versions of the same package installed on
debian-based multiarch systems - e.g. i386 alongside amd64. This means
that when getting the package status through dpkg-query we'd get a
duplicated string:
% dpkg-query -f '$${db:Status-Status}' -W libssl-dev
$installed$installed
% dpkg -l | grep libssl-dev
ii libssl-dev:amd64 3.0.11-1~deb12u2 amd64 Secure Sockets Layer toolkit - development files
ii libssl-dev:i386 3.0.11-1~deb12u2 i386 Secure Sockets Layer toolkit - development files
The fix is to explicitely check for the main architecture and, in case
for noarch (or rather all-arch in debian terms) packages, check for
special :all architecture as well.
|
|
This makes it reflect current reality.
NOTE: This removes the bit about updating the changes.xml file.
For me that has been a constant source of problems. Especially when it
hasn't been done as a separate commit (makes reverting changes harder
due to this file being constantly re-worked).
This file is also usually re-worked at release time, with the re-wording
and re-ordering of items.
In my experience it is much better to leave the updating of this file to
release time when you can use 'git shortlog -e <prev release>..' as the
source for adding entries to the changelog.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Re-flow text to wrap a little before the 80 column mark, this improves
the reading/editing experience in standard ANSI terminals and also
improves the diffing by reducing the amount of wrapping that then
occurs with the +/- additions.
This is a preparatory patch for future editing of this document.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This just sets some basic file properties; character encoding, line
endings, tabs vs spaces etc and is _not_ a replacement for a code
formatter like indent(1) or clang-format.
Link: <https://editorconfig.org/>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Added fuzzing targets:
1. djb hash
2. murmur hash2
3. parse
4. sha1
5. uri decode, uri encode
6. utf8 casecmp
7. websocket base64 encode
8. websocket frame
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Added 'nxt_conf_json_length' check for extra coverage.
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
False positive bug in harness due to improper use of the internal API.
Fixes: a93d878 ("fuzzing: add fuzzing targets")
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
* default behavior is now a read write application mount
* use can specify a flag (-r) to mount app dir as read only
Signed-off-by: Ava Hahn <a.hahn@f5.com>
|
|
|
|
|
|
|
|
Signed-off-by: Ava Hahn <a.hahn@f5.com>
|
|
Signed-off-by: Ava Hahn <a.hahn@f5.com>
|
|
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.64 to 0.10.66.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.64...openssl-v0.10.66)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
|
|
Bumps <https://github.com/github/codeql-action> from 2 to 3.
Link: Release notes <https://github.com/github/codeql-action/releases>
Link: Changelog <https://github.com/github/codeql-action/blob/main/CHANGELOG.md>
Link: Commits <https://github.com/github/codeql-action/compare/v2...v3>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Fixes: 965fc94e ("fuzzing: add fuzzing infrastructure in build system")
Fixes: 5b65134c ("fuzzing: add a basic README")
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
There are multiple false positive bugs in harness due to improper
use of the internal API.
Fixes: a93d878e ("fuzzing: add fuzzing targets")
Signed-off-by: Arjun <pkillarjun@protonmail.com>
[ Removed private links - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Fixes: 707f4ef8 ("status: Show list of loaded language modules")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
As the comment for 'Memory-only buffers' says
"... it is equal to offsetof(nxt_buf_t, file.pos)"
and
"... that is it is nxt_buf_t without file and mmap part"
Those are at odds with each other, 'file.pos' comes _after_ 'file' in
the nxt_buf_t structure.
Fix the 'offset()' bit of the comment to reflect that and to match the
relevant macro
#define NXT_BUF_MEM_SIZE offsetof(nxt_buf_t, file)
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Now that the `/status` endpoint returns a list of loaded language
modules, e.g
{
"modules": {
"python": {
"version": "3.12.2",
"lib": "/opt/unit/modules/python.unit.so"
},
...
...
}
This broke 'test/test_status.py' in a number of ways
1) The check for all the object values being 0 at startup is no longer
true with the modules section.
2) The find_diffs() check broke trying to subtract strings from
strings.
So don't include the 'modules' section in the check_zeros() check and in
the find_diffs() check, if we're dealing with strings do a basic
compare returning that value instead.
[ Commit message - Andrew ]
Co-developed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When querying the '/status' node in the control API, display the list of
currently loaded modules.
So we now get something like
{
"modules": {
"python": [
{
"version": "3.12.3",
"lib": "/opt/unit/modules/python.unit.so"
},
{
"version": "3.12.1",
"lib": "/opt/unit/modules/python-3.12.1.unit.so"
}
],
"wasm": {
"version": "0.1",
"lib": "/opt/unit/modules/wasm.unit.so"
},
"wasm-wasi-component": {
"version": "0.1",
"lib": "/opt/unit/modules/wasm_wasi_component.unit.so"
}
},
...
}
This can be useful for debugging to show exactly what modules Unit has
loaded _and_ from where.
Closes: https://github.com/nginx/unit/issues/1343
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The nxt_app_lang_module_t structure contains various bits of information
as obtained from the nxt_app_module_t structure that language modules
define.
One bit of information that is in the nxt_app_module_t but not in the
nxt_app_lang_module_t is the language module name.
Having this name flowed through will be useful for displaying the loaded
language modules in the /status endpoint.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
In nxt_status_get() call nxt_conf_set_member() multiple times to set the
main /status json sections.
Previously this used hard coded values, 0, 1, 2 etc, if you wanted to
change the order or insert new sections it could mean renumbering all
these.
Instead use a variable to track this index which starts at 0 and is
simply incremented in each call of nxt_conf_set_member().
Currently this is only for the main outer sections, but can be
replicated for inner sections if required.
This is a preparatory patch for adding a new "modules" section at the
top.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is yet more missed constification, due in this case to me searching
for 'static nxt_str_t ' but these only having a single space after the
type...
Anyway no problem, this can be a preparatory patch for adding further
/status information...
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
FreeBSD introduced sha512sum binary in version 14, but with slightly
incompatible flags as compared to Linux version. This change makes it
work in both worlds.
|
|
This commit refactors the CLI code to accept
multiple instances of the control socket flag.
All subcommands except for edit and save now
support being run against multiple specified
instances of unitd.
* control_socket_addresses CLI field is now a vector
* centralize error related logic into the error module
* wait_for_socket now returns a vector of sockets. all
sockets in vector are waited upon and validated
* extraneous code is removed
* applications, execute, import, listeners, and status
commands all run against N control sockets now
* edit and save commands return error when run against
a single control socket
Signed-off-by: Ava Hahn <a.hahn@f5.com>
|
|
Normally when the language modules are built, they are built directly
into the build/lib/unit/modules/ directory.
This then allows Unit to find them without being installed. This is
useful for things like the pytests.
This wasn't happening for the wasm-wasi-component language module. So we
now copy it over and give it the right name as part of the make/build
process.
Reported-by: Andrei Zeliankou <zelenkov@nginx.com>
Fixes: 4e6d7e876 ("Wasm-wc: Wire it up to the build system")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
|
|
|
|
The default on Ubuntu 24.04 and newer is now -D_FORTIFY_SOURCE=3 which
clashes with our definition. We shouldnt be setting it for Ubuntus
anyway since _FORTIFY_SOURCE=2 for older distros is already handled by
the defaults in their gcc builds.
|
|
|
|
|
|
These somehow got missed in my previous constification patches...
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Previously, the certificate deletion only handled string type
certificates, causing issues when certificates were specified
as an array in the configuration.
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Add the following tests cases:
1. When "factory" key is used inside the "targets" option.
2. When "factory" key is used at the root level of python application
config.
3. When factory returns invalid callable or When factory is invalid
callable
Link: <https://github.com/nginx/unit/pull/1336>
[ Commit subject & message formatting tweaks - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Adds support for the app factory pattern to the Python language module.
A factory is a callable that returns a WSGI or ASGI application object.
Unit does not support passing arguments to factories.
Setting the `factory` option to `true` instructs Unit to treat the
configured `callable` as a factory.
For example:
"my-app": {
"type": "python",
"path": "/srv/www/",
"module": "hello",
"callable": "create_app",
"factory": true
}
This is similar to other WSGI / ASGI servers. E.g.,
$ uvicorn --factory hello:create_app
$ gunicorn 'hello:create_app()'
The factory setting defaults to false.
Closes: https://github.com/nginx/unit/issues/1106
Link: <https://github.com/nginx/unit/pull/1336#issuecomment-2179381605>
[ Commit message - Dan / Minor code tweaks - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
njs changed strings API so now instead of njs_vm_value_string_set() used
njs_vm_value_string_create() as a drop-in replacement.
Link: <https://github.com/nginx/njs/commit/5730d5ffe23a4965c001d873695d22005fcfa588>
|
