| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Don't try and run the tests that require njs if it isn't enabled.
Closes: https://github.com/nginx/unit/issues/1411
Fixes: 43c4bfdcd ("tests: "if" option in http route match")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This will catch changes to the likes of wasmtime and njs.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
With commit 9998918db ("Packages: bump wasmtime to 24.0.0 and
wasi-sysroot to 24.0.") the paths to the wasmtime C API include and lib
directories changed which broke the wasm ci tests.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Wasm module is now not built for Amazon Linux 2, Debian 11 and Ubuntu
2.0.04, since it requires cmake version newer than what's available on
those OSes. wasm-wasi-component is not affected.
|
|
- Adds `unitctl/` prefix to tags generated by manual workflow runs.
Previously, only release titles (but not tags) were prefixed.
- Omits superfluous `name` field; falls back to `tag` when absent.
- Removes unnecessary conditional from `prelease` field.
This results in the following tagging / releasing behavior:
1. Running manually creates a pre-release and tags it `unitctl/VERSION`
2. Pushing a tag formatted like `x.y.z` creates a normal release
Refines: 3501a50ffb93756e145295021ff9313ac77f1ba9
|
|
[ Tweaked subject - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
While the C based wasm language module inherits the process environment
the Rust based wasm-wasi-component language module did not.
One upshot of this is that with wasm-wasi-component you don't get access
to any environment variables specified in the Unit configuration.
wasm-wasi-component was based on wasmtime 17.0.0. This capability wasn't
added to the wasmtime-crate until version 20.0.0.
Now that wasm-wasi-component has been updated to a newer wasmtime-crate
we can enable this functionality.
Closes: https://github.com/nginx/unit/issues/1312
[ Commit message - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Ava Hahn <a.hahn@f5.com>
|
|
We now have tests for this module via commit cad6aed52 ("Tests: initial
"wasm-wasi-component" test").
We need to install cargo-component for this test target.
Also the only way I found I could get this test to run was by running as
non-root. The issue I was seeing was that despite cargo being installed
into /home/runner/.cargo/bin *and* that being in the path, it kept
claiming it couldn't find cargo. E.g.
$ sudo -E echo $PATH
Showed /home/runner/.cargo/bin in there.
$ sudo -E /home/runner/.cargo/bin/cargo -V
Worked.
$ sudo -E cargo -V
cargo command not found.
(Also other oddities, despite claiming to be using bash, it couldn't
find shell builtins like 'hash' and 'export', perhaps some Ubuntu
weirdness...)
However, no problem, there is *no* need for it run as root anyway so
result!
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Rename this to 'test_wasm-wasi-component.py' to match the language
module name and the name as used in the CI.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Changes are afoot... wasm32-wasi has been renamed wasm32-wasip1, there
is also a wasm32-wasip2 (seems not yet fully realised) and wasm32-wasi
is being kept clear for an eventual WASI 1.0 release.
cargo-component targets wasm32-wasip1 by default and adapts the module
to the preview2 version of WASI supported by the component model.
This means that the component is now found under
target/wasm32-wasip1/...
Link: <https://doc.rust-lang.org/nightly/rustc/platform-support/wasm32-wasip1.html>
Link: <https://github.com/bytecodealliance/cargo-component/blob/main/README.md#wasi-support>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
We can now get list objects from the /status endpoint in the case of
having different versions of the same language module.
That led to this error
> return d1 - d2
E TypeError: unsupported operand type(s) for -: 'list' and 'list'
We already cover a similar case for when we have simple strings so add
this to that.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This directory is needed for contribs to function.
|
|
Probably not needed now...
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is no longer needed since GitHub is our primary repository now.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Closes: https://github.com/nginx/unit/issues/1352
|
|
|
|
|
|
This was found via ASan.
Given a listener address like ":" (or any address where the first
character is a colon) we can end up under-reading the addr->start
buffer here
if (nxt_slow_path(*(buf + length - 1) == '.')) {
due to length (essentially the position of the ":" in the string) being
0.
Seeing as any address that starts with a ":" is invalid Unit config
wise, we should simply reject the address if length == 0 in
nxt_sockaddr_inet_parse().
Link: <https://clang.llvm.org/docs/AddressSanitizer.html>
Signed-off-by: Arjun <pkillarjun@protonmail.com>
[ Commit message - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
False positive bug in harness due to improper use of the internal API.
Fixes: a93d878 ("fuzzing: add fuzzing targets")
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Add entries for the new 'backlog' and 'listen_threads' config options
introduced in commits 57c88fd40 ("router: Make the number of router
threads configurable") and 76489fb7e ("conf, router: Make the listen(2)
backlog configurable").
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
- Missing end quotes
- Wrong indentation
- Missing langMod parameter definition
- Wrong type specification
Fixes: ae4795aa1 ("docs/openapi: Add entries for the new /status/modules endpoint")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
unitctl makes use of 'docs/unit-openapi.yaml' so be sure to run these
checks if that file changes.
Fixes: 6d0880c99 ("Add unitctl build and release CI")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
This feature allows users to specify conditions to check if one
route is matched. It is used the same way as the "if" option in
the access log.
Example:
{
"match": {
"if": "`${headers['User-Agent'].split('/')[0] == 'curl'}`"
},
"action": {
"return": 204
}
}
|
|
|
|
This nxt_tstr_cond_t will be reused for the feature of adding "if"
option to the "match" object. The two "if" options have the same usage.
|
|
|
|
|
|
|
|
|
|
|
|
Initially, variable query was designed to accomodate both synchronous
and asynchronous operations. However, upon consideration of actual
requirements, we recognized that asynchronous support was not needed.
The refactoring ensures that the success or failure of the variable
query operation is now directly indicated by its return value. This
change streamlines the function's usage and enhances code clarity,
as it facilitates immediate error handling without the need for
asynchronous callbacks or additional error checking functions.
Note the patch only works for Unit native variables but not njs
variables.
|
|
@oopsoop2 on GitHub reported a performance issue related to the default
listen(2) backlog size of 511 on nginx. They found that increasing it
helped, nginx has a config option to configure this.
They would like to be able to do the same on Unit (which also defaults
to 511 on some systems). This seems reasonable.
NOTE: On Linux before commit 97c15fa38 ("socket: Use a default listen
backlog of -1 on Linux") we defaulted to 511. Since that commit we
default to the Kernels default, which before 5.4 is 128 and after is
4096.
This adds a new per-listener 'backlog' config option, e.g
{
"listeners": {
"[::1]:8080": {
"pass": "routes",
"backlog": 1024
},
}
...
}
This doesn't effect the control socket.
Closes: https://github.com/nginx/unit/issues/1384
Reported-by: <https://github.com/oopsoop2>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
On FreeBSD, OpenBSD & macOS we use a default listen(2) backlog of -1
which means use the OS's default value.
On Linux (and others) we used a hard coded value of 511, presumably due
to this comment
/* Linux, Solaris, and NetBSD treat negative value as 0. */
On Linux (at least since 2.4), this is wrong, Linux treats -1 (and so
on) as use the OS's default (net.core.somaxconn). See this code in
net/socket.c::__sys_listen()
if ((unsigned int)backlog > somaxconn)
backlog = somaxconn;
On Linux prior to 5.4 somaxconn defaulted to 128, since 5.4 it defaults
to 4096.
We've had complaints that a listen backlog of 511 is too small. This
would help in those cases.
Unless they are on an old Kernel, in which case it's worse, but then the
plan is to also make this configurable. This would effect RHEL 8, which
is based on 4.10, however they seem to set somaxconn to 2048, so that's
fine.
Another advantage of using -1 is that we will automatically keep up to
date with the kernels default value.
Before this change
$ ss -tunxlp | grep unit
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
u_str LISTEN 0 511 /opt/unit/control.unit.sock.tmp 4302333 * 0 users:(("unitd",pid=18290,fd=6),("unitd",pid=18289,fd=6),("unitd",pid=18287,fd=6))
tcp LISTEN 0 511 127.0.0.1:8080 0.0.0.0:* users:(("unitd",pid=18290,fd=12))
tcp LISTEN 0 511 [::1]:8080 [::]:* users:(("unitd",pid=18290,fd=11))
After
$ ss -tunxlp | grep unit
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
u_str LISTEN 0 4096 /opt/unit/control.unit.sock.tmp 5408464 * 0 users:(("unitd",pid=132442,fd=6),("unitd",pid=132441,fd=6),("unitd",pid=132439,fd=6))
tcp LISTEN 0 4096 127.0.0.1:8080 0.0.0.0:* users:(("unitd",pid=132442,fd=12))
tcp LISTEN 0 4096 [::1]:8080 [::]:* users:(("unitd",pid=132442,fd=11))
Link: <https://github.com/nginx/unit/issues/1384>
Link: <https://lore.kernel.org/netdev/20191030163620.140387-1-edumazet@google.com/>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Unit generally creates an extra number of router threads (to handle
client connections, not incl the main thread) to match the number of
available CPUs.
There are cases when this can go wrong, e.g on a high CPU count machine
and Unit is being effectively limited to a few CPUs via the cgroups cpu
controller. So Unit may create a large number of router threads when
they are only going to effectively run on a couple of CPUs or so.
There may be other cases where you would like to tweak the number of
router threads, depending on your workload.
As it turns out it looks like it was intended to be made configurable
but was just never hooked up to the config system.
This adds a new '/settings/listen_threads' config option which can be
set like
{
"listen": {
...
},
"settings": {
"listen_threads": 2,
...
},
...
}
Before this patch (on a four cpu system)
$ ps -efL | grep router
andrew 419832 419829 419832 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 5 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 445145 0 5 03:31 pts/10 00:00:00 unit: router
andrew 419832 419829 445146 0 5 03:31 pts/10 00:00:00 unit: router
After, with a threads setting of 2
$ ps -efL | grep router
andrew 419832 419829 419832 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419833 0 3 Aug12 pts/10 00:00:00 unit: router
andrew 419832 419829 419834 0 3 Aug12 pts/10 00:00:00 unit: router
Closes: https://github.com/nginx/unit/issues/1042
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
At startup, the unit router process creates a number of threads, it
tries to create the same number of threads (not incl the main thread) as
there are 'cpus' in the system.
On Linux the number of available cpus is determined via a call to
sysconf(_SC_NPROCESSORS_ONLN);
in a lot of cases this produces the right result, i.e. on a four cpu
system this will return 4.
However this can break down if unit has been restricted in the cpus it's
allowed to run on via something like cpuset()'s and/or
sched_setaffinity(2).
For example, on a four 'cpu' system, starting unit will create an extra
4 router threads
$ /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 234102 234099 234102 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234103 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234104 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234105 0 5 17:00 pts/10 00:00:00 unit: router
andrew 234102 234099 234106 0 5 17:00 pts/10 00:00:00 unit: router
Say we want to limit unit to two cpus, i.e.
$ taskset -a -c 2-3 /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 235772 235769 235772 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235773 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235774 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235775 0 5 17:08 pts/10 00:00:00 unit: router
andrew 235772 235769 235776 0 5 17:08 pts/10 00:00:00 unit: router
So despite limiting unit to two cpus
$ grep Cpus_allowed_list /proc/235772/status
Cpus_allowed_list: 2-3
It still created 4 threads, probably not such an issue in this case, but
if we had a 64 'cpu' system and wanted to limit unit two cpus, then we'd
have 64 threads vying to run on two cpus and with our spinlock
implementation this can cause a lot of thread scheduling and congestion
overhead.
Besides, our intention is currently to create nr router threads == nr
cpus.
To resolve this, on Linux at least, this patch makes use of
sched_getaffinity(2) to determine what cpus unit is actually allowed to
run on.
We still use the result of
sysconf(_SC_NPROCESSORS_ONLN);
as a fallback, we also use its result to allocate the required cpuset
size (where sched_getaffinity() will store its result) as the standard
cpu_set_t only has space to store 1023 cpus.
So with this patch if we try to limit unit to two cpus we now get
$ taskset -a -c 2-3 /opt/unit/sbin/unitd
$ ps -efL | grep router
andrew 236887 236884 236887 0 3 17:20 pts/10 00:00:00 unit: router
andrew 236887 236884 236888 0 3 17:20 pts/10 00:00:00 unit: router
andrew 236887 236884 236889 0 3 17:20 pts/10 00:00:00 unit: router
This also applies to the likes of docker, if you run docker with the
--cpuset-cpus="" option, unit will now create a number of router threads
that matches the cpu count specified.
Perhaps useful if you are running a number of unit docker instances on a
high cpu count machine.
Link: <https://github.com/nginx/unit/issues/1042>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This will help to better determine the number of router threads to
create in certain situations.
Unlike sysconf(_SC_NPROCESSORS_ONLN) this takes into account per-process
cpu allowed masks as set by sched_setaffinity(2)/cpusets etc.
So while a system may have 64 on-line cpu's, Unit itself may be limited
to using just four of them in which case we should create four extra
router threads, not sixty-four!
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Unit now shows the loaded language modules under /status/modules. This
functionality was added in commit 707f4ef82 ("status: Show list of
loaded language modules").
[ Commit message - Andrew ]
Co-developed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This was renamed to be more accurate as /status is not just about
statistics, as it also now shows the loaded language modules.
[ Commit message - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Seems the README was never updated to account for WebAssembly...
Reported-by: Palmese Davide Mattia <https://github.com/PalmeseMattia>
Closes: https://github.com/nginx/unit/issues/1376
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
It's possible to have two versions of the same package installed on
debian-based multiarch systems - e.g. i386 alongside amd64. This means
that when getting the package status through dpkg-query we'd get a
duplicated string:
% dpkg-query -f '$${db:Status-Status}' -W libssl-dev
$installed$installed
% dpkg -l | grep libssl-dev
ii libssl-dev:amd64 3.0.11-1~deb12u2 amd64 Secure Sockets Layer toolkit - development files
ii libssl-dev:i386 3.0.11-1~deb12u2 i386 Secure Sockets Layer toolkit - development files
The fix is to explicitely check for the main architecture and, in case
for noarch (or rather all-arch in debian terms) packages, check for
special :all architecture as well.
|
|
This makes it reflect current reality.
NOTE: This removes the bit about updating the changes.xml file.
For me that has been a constant source of problems. Especially when it
hasn't been done as a separate commit (makes reverting changes harder
due to this file being constantly re-worked).
This file is also usually re-worked at release time, with the re-wording
and re-ordering of items.
In my experience it is much better to leave the updating of this file to
release time when you can use 'git shortlog -e <prev release>..' as the
source for adding entries to the changelog.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Re-flow text to wrap a little before the 80 column mark, this improves
the reading/editing experience in standard ANSI terminals and also
improves the diffing by reducing the amount of wrapping that then
occurs with the +/- additions.
This is a preparatory patch for future editing of this document.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This just sets some basic file properties; character encoding, line
endings, tabs vs spaces etc and is _not_ a replacement for a code
formatter like indent(1) or clang-format.
Link: <https://editorconfig.org/>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Added fuzzing targets:
1. djb hash
2. murmur hash2
3. parse
4. sha1
5. uri decode, uri encode
6. utf8 casecmp
7. websocket base64 encode
8. websocket frame
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Added 'nxt_conf_json_length' check for extra coverage.
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
False positive bug in harness due to improper use of the internal API.
Fixes: a93d878 ("fuzzing: add fuzzing targets")
Signed-off-by: Arjun <pkillarjun@protonmail.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
