| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When running as a daemon. unit currently sets umask(0), i.e no umask.
This is resulting in various directories being created with a mode of
0777, e.g
rwxrwxrwx
this is currently affecting cgroup and rootfs directories, which are
being created with a mode of 0777, and when running as a daemon as there
is no umask to restrict the permissions.
This also affects the language modules (the umask is inherited over
fork(2)) whereby unless something explicitly sets a umask, files and
directories will be created with full permissions, 0666 (rw-rw-rw-)/
0777 (rwxrwxrwx) respectively.
This could be an unwitting security issue.
My original idea was to just remove the umask(0) call and thus inherit
the umask from the executing shell/program.
However there was some concern about just inheriting whatever umask was
in effect.
Alex suggested that rather than simply removing the umask(0) call we
change it to a value of 022 (which is a common default), which will
result in directories and files with permissions at most of 0755
(rwxr-xr-x) & 0644 (rw-r--r--).
If applications need some other umask set, they can (as they always have
been able to) set their own umask(2).
Suggested-by: Alejandro Colomar <alx.manpages@gmail.com>
Reviewed-by: Liam Crilly <liam@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When using the 'rootfs' isolation option, by default a tmpfs filesystem
is mounted on tmp/. Currently this is mounted with a mode of 0777, i.e
drwxrwxrwx. 3 root root 60 Feb 22 11:56 tmp
however this should really have the sticky bit[0] set (as is per-normal for
such directories) to prevent users from having free reign on the files
contained within.
What we really want is it mounted with a mode of 01777, i.e
drwxrwxrwt. 3 root root 60 Feb 22 11:57 tmp
[0]: To quote inode(7)
"The sticky bit (S_ISVTX) on a directory means that a file in that
directory can be renamed or deleted only by the owner of the file, by
the owner of the directory, and by a privileged process."
Reviewed-by: Liam Crilly <liam@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
|
|
While at it, fixed changelogs generation for Python 3.10 as well.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
pkg-config package is named differently on supported rpm-based systems:
- Amazon Linux 2 has pkgconfig
- Fedora has pkgconf-pkg-config
- RHEL 7 has pkgconfig
- RHEL 8 and 9 have pkgconfig-pkg-config
What they share in common is they all provide 'pkgconfig', which we can
use in the spec file so we don't have to specify it per-OS.
|
|
|
|
|
|
Since the previous commit, nxt_getpid() is only ever aliased to
getpid(2).
nxt_getpid() was only used once in the code, while there are multiple
direct uses of getpid(2)
$ grep -r "getpid()" src/
src/nxt_unit.c: nxt_unit_pid = getpid();
src/nxt_process.c: nxt_pid = nxt_getpid();
src/nxt_process.c: nxt_pid = getpid();
src/nxt_lib.c: nxt_pid = getpid();
src/nxt_process.h:#define nxt_getpid() \
src/nxt_process.h:#define nxt_getpid() \
src/nxt_process.h: getpid()
Just remove it and convert the _single_ instance of nxt_getpid() to
getpid(2).
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
This script combines the old setup-unit (as the repo-config command),
with new functionality, to provide an easy welcome website for
first-time users, and also some more commands that are useful for
administrating a running unitd(8) instance.
Suggested-by: Liam Crilly <liam@nginx.com>
Cc: Konstantin Pavlov <thresh@nginx.com>
Cc: Artem Konev <a.konev@f5.com>
Cc: Timo Start <t.stark@nginx.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
|
|
This patch gives users the option to set a `"prefix"` attribute
for Python applications, either at the top level or for specific
`"target"`s. If the attribute is present, the value of `"prefix"`
must be a string beginning with `"/"`. If the value of the `"prefix"`
attribute is longer than 1 character and ends in `"/"`, the
trailing `"/"` is stripped.
The purpose of the `"prefix"` attribute is to set the `SCRIPT_NAME`
context value for WSGI applications and the `root_path` context
value for ASGI applications, allowing applications to properly route
requests regardless of the path that the server uses to expose the
application.
The context value is only set if the request's URL path begins with
the value of the `"prefix"` attribute. In all other cases, the
`SCRIPT_NAME` or `root_path` values are not set. In addition, for
WSGI applications, the value of `"prefix"` will be stripped from
the beginning of the request's URL path before it is sent to the
application.
Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Artem Konev <artem.konev@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
|
|
Fixes https://github.com/nginx/unit/issues/728
Refs https://github.com/nginx/unit/issues/718
|
|
|
|
|
|
Added tests for the "large_header_buffer_size" and
"large_header_buffers" configuration options.
|
|
This is an extension to the previous commit, which made
large_header_buffer_size a valid configuration setting.
This commit makes a related value, large_header_buffers, a valid
configuration setting.
While large_header_buffer_size effectively limits the maximum size of
any single header (although unit will try to pack multiple headers into
a buffer if they wholly fit).
large_header_buffers limits how many of these 'large' buffers are
available. It makes sense to also allow this to be user set.
large_header_buffers is already set by the configuration system in
nxt_router.c it just isn't set as a valid config option in
nxt_conf_validation.c
With this change users can set this option in their config if required
by the following
"settings": {
"http": {
"large_header_buffers": 8
}
},
It retains its default value of 4 if this is not set.
NOTE: This is being released as undocumented and subject to change as it
exposes internal workings of unit.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
@JanMikes and @tagur87 on GitHub both reported issues with long URLs
that were exceeding the 8192 byte large_header_buffer_size setting,
which resulted in a HTTP 431 error (Request Header Fields Too Large).
This can be resolved in the code by updating the following line in
src/nxt_router.c::nxt_router_conf_create()
skcf->large_header_buffer_size = 8192;
However, requiring users to modify unit and install custom versions is
less than ideal. We could increase the value, but to what?
This commit takes the option of allowing the user to set this option in
their config by making large_header_buffer_size a valid configuration
setting.
large_header_buffer_size is already set by the configuration system in
nxt_router.c it just isn't set as a valid config option in
nxt_conf_validation.c
With this change users can set this option in their config if required
by the following
"settings": {
"http": {
"large_header_buffer_size": 16384
}
},
It retains its default value of 8192 bytes if this is not set.
With this commit, without the above setting or too low a value, with a
long URL you get a 431 error. With the above setting set to a large
enough value, the request is successful.
NOTE: This setting really determines the maximum size of any single
header _value_. Also, unit will try and place multiple values
into a buffer _if_ they fully fit.
NOTE: This is being released as undocumented and subject to change as it
exposes internal workings of unit.
Closes: <https://github.com/nginx/unit/issues/521>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Hide expected alerts by default.
Silence succesfull "go build" information.
|
|
This hooks the cgroup support up to the config system so it can actually
be used.
To make use of this in unit a new "cgroup" section has been added to the
isolation configuration.
e.g
"applications": {
"python": {
"type": "python",
"processes": 5,
"path": "/opt/unit/unit-cgroup-test/",
"module": "app",
"isolation": {
"cgroup": {
"path": "app/python"
}
}
}
}
Now there are two ways to specify the path, relative, like the above
(without a leading '/') and absolute (with a leading '/').
In the above case the "python" application is placed into its own cgroup
under CGROUP_ROOT/<main unit process cgroup>/app/python. Whereas if you
specified say
"path": "/unit/app/python"
Then the python application would be placed under
CGROUP_ROOT/unit/app/python
The first option allows you to easily take advantage of any resource
limits that have already been configured for unit.
With the second method (absolute pathname) if you know of an already
existing cgroup where you'd like to place it, you can, e.g
"path": "/system.slice/unit/python"
Where system.slice has already been created by systemd and may already
have some overall system limits applied which would also apply to unit.
Limits apply down the hierarchy and lower groups can't exceed the
previous group limits.
So what does this actually look like? Lets take the unit-calculator
application[0] and have each of its applications placed into their own
cgroup. If we give each application a new section like
"isolation": {
"cgroup": {
"path": "/unit/unit-calculator/add"
}
}
changing the path for each one, we can visualise the result with the
systemd-cgls command, e.g
│ └─session-5.scope (#4561)
│ ├─ 6667 sshd: andrew [priv]
│ ├─ 6684 sshd: andrew@pts/0
│ ├─ 6685 -bash
│ ├─ 12632 unit: main v1.28.0 [/opt/unit/sbin/unitd --control 127.0.0.1:808>
│ ├─ 12634 unit: controller
│ ├─ 12635 unit: router
│ ├─ 13550 systemd-cgls
│ └─ 13551 less
├─unit (#4759)
│ └─unit-calculator (#5037)
│ ├─subtract (#5069)
│ │ ├─ 12650 unit: "subtract" prototype
│ │ └─ 12651 unit: "subtract" application
│ ├─multiply (#5085)
│ │ ├─ 12653 unit: "multiply" prototype
│ │ └─ 12654 unit: "multiply" application
│ ├─divide (#5101)
│ │ ├─ 12671 unit: "divide" prototype
│ │ └─ 12672 node divide.js
│ ├─sqroot (#5117)
│ │ ├─ 12679 unit: "sqroot" prototype
│ │ └─ 12680 /home/andrew/src/unit-calculator/sqroot/sqroot
│ └─add (#5053)
│ ├─ 12648 unit: "add" prototype
│ └─ 12649 unit: "add" application
We used an absolute path so the cgroups will be created relative to the
main cgroupfs mount, e.g /sys/fs/cgroup
We can see that the main unit processes are in the same cgroup as the
shell from where they were started, by default child process are placed
into the same cgroup as the parent.
Then we can see that each application has been placed into its own
cgroup under /sys/fs/cgroup
Taking another example of a simple 5 process python application, with
"isolation": {
"cgroup": {
"path": "app/python"
}
}
Here we have specified a relative path and thus the python application
will be placed below the existing cgroup that contains the main unit
process. E.g
│ │ │ ├─app-glib-cinnamon\x2dcustom\x2dlauncher\x2d3-43951.scope (#90951)
│ │ │ │ ├─ 988 unit: main v1.28.0 [/opt/unit/sbin/unitd --no-daemon]
│ │ │ │ ├─ 990 unit: controller
│ │ │ │ ├─ 991 unit: router
│ │ │ │ ├─ 43951 xterm -bg rgb:20/20/20 -fg white -fa DejaVu Sans Mono
│ │ │ │ ├─ 43956 bash
│ │ │ │ ├─ 58828 sudo -i
│ │ │ │ ├─ 58831 -bash
│ │ │ │ └─app (#107351)
│ │ │ │ └─python (#107367)
│ │ │ │ ├─ 992 unit: "python" prototype
│ │ │ │ ├─ 993 unit: "python" application
│ │ │ │ ├─ 994 unit: "python" application
│ │ │ │ ├─ 995 unit: "python" application
│ │ │ │ ├─ 996 unit: "python" application
│ │ │ │ └─ 997 unit: "python" application
[0]: <https://github.com/lcrilly/unit-calculator>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This commit enables the building of the cgroup code. This is only built
when the cgroupv2 filesystem is found.
If cgroupv2 support is found then
cgroupv2: .................. YES
will be printed by ./configure
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This commit hooks into the cgroup infrastructure added in the previous
commit to create per-application cgroups.
It does this by adding each "prototype process" into its own cgroup,
then each child process inherits its parents cgroup.
If we fail to create a cgroup we simply fail the process. This behaviour
may get enhanced in the future.
This won't actually do anything yet. Subsequent commits will hook this
up to the build and config systems.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Firstly, this is not to be confused with CLONE_NEWCGROUP which unit
already supports and is related to namespaces. To re-cap, namespaces
allow processes to have different views of various parts of the system
such as filesystem mounts, networking, hostname etc.
Whereas cgroup[0] is a Linux kernel facility for collecting a bunch of
processes together to perform some task on the group as a whole, for
example to implement resource limits.
There are two parts to cgroup, the core part of organising processes
into a hierarchy and the controllers which are responsible for enforcing
resource limits etc.
There are currently two versions of the cgroup sub-system, the original
cgroup and a version 2[1] introduced in 3.16 (August 2014) and marked
stable in 4.5 (March 2016).
This commit supports the cgroup V2 API and implements the ability to
place applications into their own cgroup on a per-application basis.
You can put them each into their own cgroup or you can group some
together. The ability to set resource limits can easily be added in
future.
The initial use case of this would be to aid in observability of unit
applications which becomes much easier if you can just monitor them on a
per cgroup basis.
One thing to note about cgroup, is that unlike namespaces which are
controlled via system calls such as clone(2) and unshare(2), cgroups are
setup and controlled through the cgroupfs pseudo-filesystem.
cgroup is Linux only and this support will only be enabled if configure
finds the cgroup2 filesystem mount, e.g
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
The cgroups are removed on shutdown or as required on reconfiguration.
This commit just adds the basic infrastructure for using cgroups within
unit. Subsequent commits will wire up this support.
It supports creating cgroups relative to the main cgroup root and also
below the cgroup of the main unit process.
[0]: <https://man7.org/linux/man-pages/man7/cgroups.7.html>
[1]: <https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html>
Cc: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Add simple wrapper functions for fopen(3) and fclose(3) that are
somewhat akin to the nxt_file_open() and nxt_file_close() wrappers that
log errors.
Suggested-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Andrei reported an issue with building unit when using '-O0' with GCC
producing the following compiler errors
cc -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -O0 -I src -I build \
\
\
-o build/src/nxt_unit.o \
-MMD -MF build/src/nxt_unit.dep -MT build/src/nxt_unit.o \
src/nxt_unit.c
src/nxt_unit.c: In function ‘nxt_unit_log’:
src/nxt_unit.c:6601:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6601 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6582:22: note: ‘msg’ declared here
6582 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
src/nxt_unit.c: In function ‘nxt_unit_req_log’:
src/nxt_unit.c:6645:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6645 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6625:35: note: ‘msg’ declared here
6625 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
cc1: all warnings being treated as errors
The above was reproduced with
$ ./configure --cc-opt=-O0 && ./configure python && make -j4
This warning doesn't happen on clang (15.0.4) or GCC (8.3) and seems to
have been introduced in GCC 11. The above is from GCC (12.2.1, Fedora
37).
The trigger of this GCC issue is actually part of a commit I introduced
a few months back to constify some function parameters and it seems the
consensus for how to resolve this problem is to simply remove the const
qualifier from the *end parameter to nxt_unit_snprint_prefix().
Reported-by: Andrei Zeliankou <zelenkov@nginx.com>
Link: <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100417>
Link: <https://github.com/samtools/htslib/pull/1285>
Link: <https://gcc.gnu.org/gcc-11/changes.html>
Fixes: 4418f99 ("Constified numerous function parameters.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When using SYS_clone we used the getpid kernel system call directly via
syscall(SYS_getpid) to avoid issues with cached pids.
However since we are now only using fork(2) (+ unshare(2) for
namespaces) we no longer need to call the kernel getpid directly as the
fork(2) will ensure the cached pid is invalidated.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Since the previous commit, this is no longer used.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
On GitHub, @razvanphp & @hbernaciak both reported issues running the
APCu PHP module under Unit.
When using this module they were seeing errors like
'apcu_fetch(): Failed to acquire read lock'
However when running APCu under php-fpm, everything was fine.
The issue turned out to be due to our use of SYS_clone breaking the
pthreads(7) API used by APCu. Even if we had been using glibc's
clone(2) wrapper we would still have run into problems due to a known
issue there.
Essentially the problem is when using clone, glibc doesn't update the
TID cache, so the child ends up having the same TID as the parent and
that is used in various parts of pthreads(7) such as in the various
locking primitives, so when APCu was grabbing a lock it ended up using
the TID of the main unit process (rather than that of the php
application processes that was grabbing the lock).
So due to the above what was happening was when one of the application
processes went to grab either a read or write lock, the lock was
actually being attributed to the main unit process. If a process had
acquired the write lock, then if a process tried to acquire a read or
write lock then glibc would return EDEADLK due to detecting a deadlock
situation due to thinking the process already held the write lock when
in fact it didn't.
It seems the right way to do this is via fork(2) and unshare(2). We
already use fork(2) on other platforms.
This requires a few tricks to keep the essence of the processes the same
as before when using clone
1) We use the prctl(2) PR_SET_CHILD_SUBREAPER option (if its
available, since Linux 3.4) to make the main unit process inherit
prototype processes after a double fork(2), rather than them being
reparented to 'init'.
This avoids needing to ^C twice to fully exit unit when running in
the foreground. It's probably also better if they maintain their
parent child relationship where possible.
2) We use a double fork(2) technique on the prototype processes to
ensure they themselves end up in a new PID namespace as PID 1 (when
CLONE_NEWPID is being used).
When using unshare(CLONE_NEWPID), the calling process is _not_
placed in the namespace (as discussed in pid_namespaces(7)). It
only sets things up so that subsequent children are placed in a PID
namespace.
Having the prototype processes as PID 1 in the new PID namespace is
probably a good thing and matches the behaviour of clone(2). Also,
some isolation tests break if the prototype process is not PID 1.
3) Due to the above double fork(2) the main unit process looses track
of the prototype process ID, which it needs to know.
To solve this, we employ a simple pipe(2) between the main unit and
prototype processes and pass the prototype grandchild PID from the
parent of the second fork(2) before exiting. This needs to be done
from the parent and not the grandchild, as the grandchild will see
itself having a PID of 1 while the main process needs its
externally visible PID.
Link: <https://www.php.net/manual/en/book.apcu.php>
Link: <https://sourceware.org/bugzilla/show_bug.cgi?id=21793>
Closes: <https://github.com/nginx/unit/issues/694>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This prctl(2) option can be used to set the "child subreaper" attribute
of the calling process. This allows a process to take on the role of
'init', which means the process will inherit descendant processes when
their immediate parent terminates.
This will be used in an upcoming commit that uses a double fork(2) +
unshare(2) to create a new PID namespace. The parent from the second
fork will terminate leaving the child process to be inherited by 'init'.
Aside from it being better to maintain the parent/child relationships
between the various unit processes, without setting this you need to ^C
twice to fully quit unit when running in the foreground after the double
fork.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Due to the need to replace our use of clone/__NR_clone on Linux with
fork(2)/unshare(2) for enabling Linux namespaces(7) to keep the
pthreads(7) API working. Let's rename NXT_HAVE_CLONE to
NXT_HAVE_LINUX_NS, i.e name it after the feature, not how it's
implemented, then in future if we change how we do namespaces again we
don't have to rename this.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This prctl(2) option is checked for in auto/isolation, unfortunately due
to a typo this feature has never been enabled.
In the auto/isolation script the feature name was down as
NXT_HAVE_PR_SET_NO_NEW_PRIVS0, which means we end up with the following
in build/nxt_auto_config.h
#ifndef NXT_HAVE_PR_SET_NO_NEW_PRIVS0
#define NXT_HAVE_PR_SET_NO_NEW_PRIVS0 1
#endif
Whereas everywhere else is checking for NXT_HAVE_PR_SET_NO_NEW_PRIVS.
This also guards the inclusion of sys/prctl.h in src/nxt_process.c which
is required by a subsequent commit.
Fixes: e2b53e1 ("Added "rootfs" feature.")
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This commit fixed the njs memory leak happened in the config validation, updating and http requests.
|
