| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
When the content length is small, optimize zlib for low memory usage.
Conversely, when the content length is large, use a similar amount of
memory within zlib, as it will improve compression, and won't hurt
significantly.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
These are based on C23's <stdbit.h>.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
With this, short responses, that is, responses with a body of up to
content_length_threshold bytes, won't be compressed. The default value
is 20, as in NGINX.
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
This is similar to atoi(3) in usage, except that:
- It's safe (doesn't have Undefined Behavior).
- It receives a nxt_str_t instead of a string.
- The base is 0, not 10.
Errors are reported with the same error codes as in strtou(3bsd), but
they are reported via errno, instead of passing a pointer to an int.
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
This function is like strtoul(3), but it rejects negative numbers,
instead of silently converting them into unsigned, which strtoul(3)
does.
Link: <https://blog.habets.se/2022/10/No-way-to-parse-integers-in-C.html>
Link: <https://softwareengineering.stackexchange.com/a/449060/332848>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Related to:
HTTP: compress: gzip
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
There are still no supported encodings. This is just infrastructure for
the next commits, which will add gzip compression.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
Filter handlers are a new handler that takes place when a buffer is
about to be sent. It filters (modifies) the contents of the buffer
in-place, so that the new contents will be sent. Several filters can be
applied in a loop.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
This will allow sending the header from a totally different point, since
the data for the call is present in the request, which is available
everywhere.
It will also allow consulting in a filter if there is a body_handler
installed. The gzip filter will need this, as it should be a no-op if
there is no body handler installed.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
nxt_min()
nxt_max()
Return the minimum/maximum of two values.
nxt_swap()
Swap the values of two variables passed by their addresses.
nxt_sizeof_array()
Return the size (in bytes) of an array.
nxt_nitems()
Return the number of elements in an array.
nxt_memberof()
Expand to a member of a structure. It uses a compound literal for
the object.
nxt_sizeof_incomplete()
Calculate the size of an incomplete type, as if sizeof() could be
applied to it.
nxt_sizeof_fam0()
Calculate the size of each element of a FAM of a structure.
nxt_sizeof_fam()
Calculate the size of a FAM of a structure.
nxt_offsetof_fam()
Calculate the offset of the nth element of the FAM from the start of
the containing structure.
nxt_sizeof_struct()
Calculate the total size of a structure containing a FAM. This
value is the one that should be used for allocating the structure.
Suggested-by: Andrew Clayton <a.clayton@nginx.com>
nxt_is_near_end()
Evaluate to true if the member is near the end of a structure. This
is only designed to be used with FAMs, to make sure that the FAM is
near the end of the structure (a zero-length array near the end of
the structure would still pass this test, but it's a reasonable
assertion to do.
Suggested-by: David Laight <David.Laight@ACULAB.COM>
nxt_is_zero_sizeof()
Evaluate to true if the size of 'x' is 0.
nxt_is_same_type()
Evaluate to true if the both arguments are compatible types.
nxt_is_same_typeof()
Evaluate to true if the types of both arguments are compatible.
nxt_is_array()
Evaluate to true if the argument is an array.
nxt_must_be()
It's like static_assert(3), but implemented as an expression. It's
necessary for writing the must_be_array() macro. It's always
evaluates to (int) 0.
nxt_must_be_array()
Statically assert that the argument is an array. It is an
expression that always evaluates to (int) 0.
nxt_must_be_zero_sizeof()
Statically assert that the argument has a size of 0.
nxt_must_be_near_end()
Statically assert that a member of a structure is near the end of
it.
Suggested-by: David Laight <David.Laight@ACULAB.COM>
nxt_must_be_fam()
Statically assert that the argument is a flexible array member
(FAM). It's an expression that always evaluates to (int) 0.
Link: <https://gustedt.wordpress.com/2011/03/14/flexible-array-member/>
Link: <https://lore.kernel.org/lkml/202308161913.91369D4A@keescook/T/>
Link: <https://inbox.sourceware.org/gcc/dac8afb7-5026-c702-85d2-c3ad977d9a48@kernel.org/T/>
Link: <https://stackoverflow.com/a/57537491>
Link: <https://github.com/shadow-maint/shadow/pull/762>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
|
|
This was inadvertently removed in 76086d6d ("Wasm: Allow to set the HTTP
response status.")
Fixes: 76086d6d ("Wasm: Allow to set the HTTP response status.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
On Github, @rlandgrebe reported an issue when trying to rewrite URLs
that contained query strings.
With the PHP language module we were in fact segfaulting (SIGSEGV) in
libphp
[93960.462952] unitd[20940]: segfault at 7f307cef6476 ip 00007f2f81a94577 sp 00007fff28a777d0 error 4 in libphp-8.2.so[7f2f818df000+2fd000] likely on CPU 0 (core 0, socket 0)
#0 0x00007f2abd494577 in php_default_treat_data (arg=1, str=0x0,
destArray=<optimized out>)
at /usr/src/debug/php-8.2.10-1.fc38.x86_64/main/php_variables.c:488
488 if (c_var && *c_var) {
(gdb) p c_var
$1 = 0x7f2bb8880676 <error: Cannot access memory at address 0x7f2bb8880676>
This was when trying to get the query string which somehow is pointing
off into the woods.
This gdb debug session when doing rewrite basically shows the core of
the issue
(gdb) x /64bs req->fields
...
0x7f7eaaaa8090: "GET"
0x7f7eaaaa8094: "HTTP/1.1"
0x7f7eaaaa809d: "::1"
0x7f7eaaaa80a1: "::1"
0x7f7eaaaa80a5: "8080"
0x7f7eaaaa80aa: "localhost"
0x7f7eaaaa80b4: "/test?q=a"
0x7f7eaaaa80be: "/test"
...
(gdb) p target_pos
$4 = (void *) 0x7f7eaaaa80b4
(gdb) p query_pos
$6 = (void *) 0x7f7eaaaa6af6
(gdb) p r->args->start
$8 = (u_char *) 0x7f7ea4002b02 "q=a HTTP/1.1\r\nHost: localhost:8080\r\nUser-Agent: curl/8.0.1\r\nAccept: */*\r\n\r\n"
(gdb) p r->target.start
$9 = (u_char *) 0x7f7ea40040c0 "/test?q=a"
That last address, 0x7f7ea40040c0, looks out of wack, it should be
smaller than r->args->start.
That results in a calculation in nxt_router_prepare_msg()
if (r->args->start != NULL) {
query_pos = nxt_pointer_to(target_pos,
r->args->start - r->target.start);
nxt_unit_sptr_set(&req->query, query_pos);
} else {
that goes negative that then is stored in req->query.offset which is a
uint32_t and so wraps backwards from UINT_MAX to give us an offset of a
little under 4GiB, hence the above invalid memory access.
All this happens due to in nxt_http_rewrite() if we have a URL with a
query string, we create a new memory allocation to store the transformed
URL and query string.
We set r->target to point to this new allocation, but we also need to
point r->args->start to the start of the query string in this new
allocation.
Reported-by: René Landgrebe <https://github.com/rlandgrebe>
Tested-by: René Landgrebe <https://github.com/rlandgrebe>
Tested-by: Liam Crilly <liam.crilly@nginx.com>
Fixes: 14d6d97b ("HTTP: added basic URI rewrite.")
Closes: <https://github.com/nginx/unit/issues/964>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The default configuration previously contained just a listeners and
applications object. Since routes is now a principle configuration object,
and a recommended way of configurating Unit, it is now included in the
default configuration.
This change benefits new users because it explicitly introduces the three
principle configuration objects which leads more intuitively to the
documentation. Experienced users may choose to ignore or delete routes.
routes is defined as an array instead of an object because this change
is designed to assist new users, where the simpler form of routes is
easier to understand.
|
|
We need to take into account the size of the nxt_unit_response_t
structure itself when calculating where to start appending data to in
memory.
Closes: <https://github.com/nginx/unit/issues/923>
Reported-by: Alejandro Colomar <alx@kernel.org>
Reviewed-by: Andrew Clayton <a.clayton@nginx.org>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This closes #871 issue on GitHub.
|
|
Starting from Node.js 15.0.0 the chunk parameter of the response.write()
can be a Uint8Array.
This closes #870 issue on GitHub.
|
|
Currently Wasm modules are limited to a 32bit address space (until at
least the memory64 work is completed). All the counters etc in the
request structure were u32's. Which matched with 32bit memory
limitation.
However there is really no need to not allow >4GiB uploads that can be
saved off to disk or some such.
To do this we just need to increase the ->content_len &
->total_content_sent members to u64's.
However because we need the request structure to have the exact same
layout on 32bit (for Wasm modules) as it does on 64bit we need to re-jig
the order of some of these members and add a four-byte padding member.
Thus the request structure now looks like on 64bit (as shown by
pahole(1))
struct nxt_wasm_request_s {
uint32_t method_off; /* 0 4 */
uint32_t method_len; /* 4 4 */
uint32_t version_off; /* 8 4 */
uint32_t version_len; /* 12 4 */
uint32_t path_off; /* 16 4 */
uint32_t path_len; /* 20 4 */
uint32_t query_off; /* 24 4 */
uint32_t query_len; /* 28 4 */
uint32_t remote_off; /* 32 4 */
uint32_t remote_len; /* 36 4 */
uint32_t local_addr_off; /* 40 4 */
uint32_t local_addr_len; /* 44 4 */
uint32_t local_port_off; /* 48 4 */
uint32_t local_port_len; /* 52 4 */
uint32_t server_name_off; /* 56 4 */
uint32_t server_name_len; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
uint64_t content_len; /* 64 8 */
uint64_t total_content_sent; /* 72 8 */
uint32_t content_sent; /* 80 4 */
uint32_t content_off; /* 84 4 */
uint32_t request_size; /* 88 4 */
uint32_t nfields; /* 92 4 */
uint32_t tls; /* 96 4 */
char __pad[4]; /* 100 4 */
nxt_wasm_http_field_t fields[]; /* 104 0 */
/* size: 104, cachelines: 2, members: 25 */
/* last cacheline: 40 bytes */
};
and the same structure (taken from unit-wasm) compiled as 32bit
struct luw_req {
u32 method_off; /* 0 4 */
u32 method_len; /* 4 4 */
u32 version_off; /* 8 4 */
u32 version_len; /* 12 4 */
u32 path_off; /* 16 4 */
u32 path_len; /* 20 4 */
u32 query_off; /* 24 4 */
u32 query_len; /* 28 4 */
u32 remote_off; /* 32 4 */
u32 remote_len; /* 36 4 */
u32 local_addr_off; /* 40 4 */
u32 local_addr_len; /* 44 4 */
u32 local_port_off; /* 48 4 */
u32 local_port_len; /* 52 4 */
u32 server_name_off; /* 56 4 */
u32 server_name_len; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
u64 content_len; /* 64 8 */
u64 total_content_sent; /* 72 8 */
u32 content_sent; /* 80 4 */
u32 content_off; /* 84 4 */
u32 request_size; /* 88 4 */
u32 nr_fields; /* 92 4 */
u32 tls; /* 96 4 */
char __pad[4]; /* 100 4 */
struct luw_hdr_field fields[]; /* 104 0 */
/* size: 104, cachelines: 2, members: 25 */
/* last cacheline: 40 bytes */
};
We can see the structures have the same layout, same size and no
padding.
We need the __pad member as otherwise I saw gcc and clang on Alpine
Linux automatically add the 'packed' attribute to the structure which
made the two structures not match.
Link: <https://github.com/WebAssembly/memory64>
Link: <https://github.com/nginx/unit-wasm>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When trying to upload files to the luw-upload-reflector demo[0] above a
certain size that would mean Unit would need to make more than two calls
to the request_handler function in the Wasm module we would get the
following error from wasmtime and the upload would stall on the third
call to the request_handler
WASMTIME ERROR: failed to call function [->wasm_request_handler]
error while executing at wasm backtrace:
0: 0x5ce2 - <unknown>!memcpy
1: 0x7df - luw_req_buf_append
at /home/andrew/src/unit-wasm/src/c/libunit-wasm.c:308:14
2: 0x3a1 - luw_request_handler
at /home/andrew/src/unit-wasm/examples/c/luw-upload-reflector.c:110:3
Caused by:
wasm trap: out of bounds memory access
This was due to ->content_off (the offset of where the actual body
content starts in the request structure/memory) being some overly large
value.
This was largely down to me being an idiot!
Before calling the loop that makes the calls to the request_handler we
would calculate the new offset, which is now just the size of the
request structure as we don't re-send all the HTTP meta data and headers
etc. However because this value is in the request structure which is in
the shared memory and we use this same memory for requests and
responses, when we make a response we overwrite this request structure
with the response structure, so our ->content_off is now some wacked out
value when we make the next call to the request_handler.
To fix this we just need to reset ->content_off each time round the
loop.
There's also no point in setting ->nfields to 0, it has the same issue
as above, but doesn't get re-used by the Wasm module anyway.
[0]: <https://github.com/nginx/unit-wasm/blob/main/examples/c/luw-upload-reflector.c>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This commit enables WebAssembly modules to set the HTTP response status
to something other than the previously hard coded '200 OK'.
To do this they can make a call to nxt_wasm_set_resp_status() providing
the required status code.
If this function isn't called the status code defaults to '200 OK'. The
WebAssembly module can also return -1 from the request_handler function
as a short cut to signal a '500 Internal Server Error'.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
As reported by @andypost on GitHub, if you try to build Unit on a system
that uses musl libc (such as Alpine Linux) with clang then you get the
following
clang -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -fstrict-aliasing -Wstrict-overflow=5 -Wmissing-prototypes -Werror -g -I src -I build/include \
\
\
-o build/src/nxt_socketpair.o \
-MMD -MF build/src/nxt_socketpair.dep -MT build/src/nxt_socketpair.o \
src/nxt_socketpair.c
In file included from src/nxt_socketpair.c:8:
src/nxt_socket_msg.h:138:17: error: comparison of integers of different signs: 'unsigned long' and 'long' [-Werror,-Wsign-compare]
cmsg = CMSG_NXTHDR(&msg, cmsg))
^~~~~~~~~~~~~~~~~~~~~~~
/usr/include/sys/socket.h:358:44: note: expanded from macro 'CMSG_NXTHDR'
__CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/nxt_socketpair.c:8:
src/nxt_socket_msg.h:177:17: error: comparison of integers of different signs: 'unsigned long' and 'long' [-Werror,-Wsign-compare]
cmsg = CMSG_NXTHDR(&msg, cmsg))
^~~~~~~~~~~~~~~~~~~~~~~
/usr/include/sys/socket.h:358:44: note: expanded from macro 'CMSG_NXTHDR'
__CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
make: *** [build/Makefile:261: build/src/nxt_socketpair.o] Error 1
GCC works fine, it seems to have some smarts so that it doesn't give
warnings on system header files.
This seems to be a long standing issue with musl libc (bad casting in
the CMSG_NXTHDR macro) and the workaround employed by several projects
is to disable the -Wsign-compare clang warning for the code in question.
So, that's what we do. We wrap the CMSG_NXTHDR macro in a function, so
we can use the pre-processor in it to selectively disable the warning.
Link: <https://github.com/dotnet/runtime/issues/16438>
Link: <https://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-devtools/breakpad/breakpad/0001-Turn-off-sign-compare-for-musl-libc.patch>
Link: <https://github.com/dotnet/corefx/blob/57ff88bb75a0/src/Native/Unix/System.Native/pal_networking.c#L811-L829>
Link: <https://patchwork.yoctoproject.org/project/oe/patch/20220407191438.3696227-1-stefan@datenfreihafen.org/>
Closes: <https://github.com/nginx/unit/issues/936>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Scripted change:
$ grep -ril recevied src/ | xargs sed -i s/recevied/received/
Reported-by: <https://github.com/jeffdafoe>
Closes: <https://github.com/nginx/unit/issues/920>
Cc: <https://github.com/meezaan>
Cc: Timo Stark <t.stark@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
Due to the sandboxed nature of WebAssembly, by default WASM modules
don't have any access to the underlying filesystem.
There is however a capabilities based mechanism[0] for allowing such
access.
This adds a config option to the 'wasm' application type;
'access.filesystem' which takes an array of directory paths that are
then made available to the WASM module. This access works recursively,
i.e everything under a specific path is allowed access to.
Example config might look like
"access" {
"filesystem": [
"/tmp",
"/var/tmp"
]
}
The actual mechanism used allows directories to be mapped differently in
the guest. But at the moment we don't support that and just map say /tmp
to /tmp. This can be revisited if it's something users clamour for.
Network sockets are another resource that may be controlled in this
manner, for example there is a wasi_config_preopen_socket() function,
however this requires the runtime to open the network socket then
effectively pass this through to the guest.
This is something that can be revisited in the future if users desire
it.
[0]:
<https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-capabilities.md>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This exposes various WebAssembly language module specific options.
The application type is "wasm".
There is a "module" option that is required, this specifies the full
path to the WebAssembly module to be run. This module should be in
binary format, i.e a .wasm file.
There are also currently eight function handlers that can be specified.
Three of them are _required_
1) request_handler
The main driving function. This may be called multiple times for a
single HTTP request if the request is larger than the shared memory.
2) malloc_handler
Used to allocate a chunk of memory at language module startup. This
memory is allocated from the WASM modules address space and is what is
sued for communicating between the WASM module (the guest) and Unit (the
host).
3) free_handler
Used to free the memory from above at language module shutdown.
Then there are the following five _optional_ handlers
1) module_init_handler
If set, called at language module startup.
2) module_end_handler
If set, called at language module shutdown.
3) request_init_handler
If set, called at the start of request. Called only once per HTTP
request.
4) request_end_handler
If set, called once all of a request has been sent to the WASM module.
5) response_end_handler
If set, called at the end of a request, once the WASM module has sent
all its headers and data.
Example config
"applications": {
"luw-echo-request": {
"type": "wasm",
"module": "/path/to/unit-wasm/examples/c/luw-echo-request.wasm",
"request_handler": "luw_request_handler",
"malloc_handler": "luw_malloc_handler",
"free_handler": "luw_free_handler",
"module_init_handler": "luw_module_init_handler",
"module_end_handler": "luw_module_end_handler",
}
}
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This adds the core of runtime WebAssembly[0] support. Future commits
will enable this in the Unit core and expose the configuration.
This introduces a new src/wasm directory for storing this source.
We are initially using Wasmtime[0] as the WebAssembly runtime, however
this has been designed with the ability to use different runtimes in
mind.
src/wasm/nxt_wasm.[ch] is the main interface to Unit.
src/wasm/nxt_rt_wasmtime.c is the Wasmtime runtime support. This is
nicely insulated from any knowledge of internal Unit workings.
Wasmtime is what loads and runs the Wasm modules.
The Wasm modules can export functions Wasmtime can call and Wasmtime can
export functions that the module can call.
We make use of both. The terminology used is that function exports are
what the Wasm module exports and function imports are what the Wasm
runtime exports to the module.
We currently have four function imports (functions exported by the
runtime to be called by the Wasm module).
1) nxt_wasm_get_init_mem_size
This allows Wasm modules to get the size of the initially allocated
shared memory. This is the size allocated at Unit startup and what the
Wasm modules can assume they have access to (in reality this shared
memory will likely be larger).
The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as
of this commit is 32MiB.
We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at
startup which is an extra 64KiB (the smallest allocation unit), this is
to allow room for the response structure and so module developers can
just assume they have the full 32MiB for their actual response.
2) nxt_wasm_send_headers
This allows WASM modules to send their headers.
3) nxt_wasm_send_response
This allows WASM modules to send their response.
4) nxt_wasm_response_end
This allows WASM modules to inform Unit they have finished sending their
response. This calls nxt_unit_request_done()
Then there are currently up to eight functions that a module can export.
Three of which are required. These function can be named anything. I'll
use the Unit configuration names to refer to them
1) request_handler
The main driving function. This may be called multiple times for a
single HTTP request if the request is larger than the shared memory.
2) malloc_handler
Used to allocate a chunk of memory at language module startup. This
memory is allocated from the WASM modules address space and is what is
sued for communicating between the WASM module (the guest) and Unit (the
host).
3) free_handler
Used to free the memory from above at language module shutdown.
Then there are the following optional handlers
1) module_init_handler
If set, called at language module startup.
2) module_end_handler
If set, called at language module shutdown.
3) request_init_handler
If set, called at the start of request. Called only once per HTTP
request.
4) request_end_handler
If set, called once all of a request has been sent to the WASM module.
5) response_end_handler
If set, called at the end of a request, once the WASM module has sent
all its headers and data.
32bits
We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version
of clang, 13+[2], do seem to have support for wasm64 as a target (which
uses a LP64 model). However it's not entirely clear if the WASI SDK
fully supports[3] this and by extension WASI libc/wasi-sysroot.
64bit support is something than can be explored more thoroughly in the
future.
As such in structures that are used to communicate between the host and
guest we use 32bit ints. Even when a single byte might be enough. This
is to avoid issues with structure layout differences between a 64bit
host and 32bit guest (I.e WASM module) and the need for various bits of
structure padding depending on host architecture. Instead everything is
4-byte aligned.
[0]: <https://webassembly.org/>
[1]: <https://wasmtime.dev/>
[2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a>
[3]: <https://github.com/WebAssembly/wasi-sdk/issues/185>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is required to actually _build_ the Wasm language module.
The nxt_wasm_app_conf_t structure consists of the modules name, e.g
wasm, then the three required function handlers followed by the five
optional function handlers.
See the next commit for details of these function handlers.
We also need to include the u.wasm union entry that provides access to
the above structure.
The bulk of the configuration infrastructure will be added in a
subsequent commit.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is the first patch in adding WebAssembly language module support.
This just adds a new NXT_APP_WASM type, required by subsequent commits.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This makes it much more clear what's what.
This is in preparation for adding WebAssembly language module support.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
No functional changes.
|
|
|
|
This commit adds the variable $response_header_NAME.
|
|
This commit is to reimplement the variables with an unknown field
such as $header_{name} to make the parsing more generic,
it's a preparation for supporting response header variables.
|
|
|
|
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
When a variable is accessed in the Unit configuration, the value is cached.
This was useful prior to the URI rewrite feature, but now that the URI (more
precisely, the request target) can be rewritten, the contents of the variable
$uri (which contains the path part of the request target, and is decoded)
should not be cached anymore, or at least the cached value should be invalidated
after a URI rewrite.
Example:
{
"rewrite": "/prefix$uri",
"share": "$uri"
}
For a request line like GET /foo?bar=baz HTTP/1.1\r\n, the expected file
served in the response would be /prefix/foo, but due to the caching issue,
Unit currently serves /foo.
|
|
Signed-off-by: synodriver <diguohuangjiajinweijun@gmail.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
[ Re-word commit subject - Andrew ]
Fixes: c4c2f90c5b53 ("Python: ASGI server introduced.")
Closes: <https://github.com/nginx/unit/issues/895>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Lifespan state is a special dict in asgi lifespan scope, which allow
applications to persist data from the lifespan cycle to request/response
handling. The scope["state"] namespace provides a place to store these
sorts of things. The server will ensure that a shallow copy of the
namespace is passed into each subsequent request/response call into the
application.
Some frameworks are already taking advantage of this feature, for
example, starlette, and without this feature they wouldn't work
properly.
Signed-off-by: synodriver <diguohuangjiajinweijun@gmail.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
[ Minor code tweaks to avoid lines > 80 chars, static a function and
re-work the PyMemberDef structure initialisation for Python <3.7
and -Wwrite-strings compatibility - Andrew ]
Tested-by: <https://github.com/synodriver>
Tested-by: <https://github.com/hawiliali>
Closes: <https://github.com/nginx/unit/issues/864>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
If we don't update the pointer before copying the request body, then we
get the behavior shown below. After this patch, "foo\n" is rightly
appended at the end of the response body.
Request:
"GET / HTTP/1.1\r\nHost: _\nContent-Length: 4\n\nfoo\n"
Response body:
"""
Hello world!
foo
est data:
Method: GET
Protocol: HTTP/1.1
Remote addr: 127.0.0.1
Local addr: 127.0.0.1
Target: /
Path: /
Fields:
Host: _
Content-Length: 4
Body:
"""
Fixes: 1bb22d1e922c ("Unit application library.")
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
We renamed the options recently, with the intention of keeping the old
names as supported but deprecated for some time, before removal. This
was done with the configure script options, but in the unitd binary, we
accidentally removed the old names, causing some unintended breakage.
Keep support for the old names, albeit with a deprecation message to
stderr, for some time, until we decide to remove them.
Fixes: 5a37171f733f ("Added default values for pathnames.")
Closes: <https://github.com/nginx/unit/issues/876>
Reported-by: El RIDO <elrido@gmx.net>
Acked-by: Liam Crilly <liam@nginx.com>
Acked-by: Artem Konev <a.konev@f5.com>
Acked-by: Timo Stark <t.stark@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
|
|
There are a couple of reports on GitHub about issues accessing Python
ASGI based applications over IPv6.
A request over IPv6 would result in an error like
2023/05/13 17:49:12 [alert] 47202#47202 [unit] #10: Python failed to create 'client' pair
2023/05/13 17:49:12 [alert] 47202#47202 [unit] Python failed to call 'loop.call_soon'
ValueError: invalid literal for int() with base 10: 'db8:1:1:1ee7:dead:beef:cafe'
The above error was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib64/python3.11/asyncio/base_events.py", line 765, in call_soon
handle = self._call_soon(callback, args, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/base_events.py", line 781, in _call_soon
handle = events.Handle(callback, args, self, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: <class 'asyncio.events.Handle'> returned a result with an exception set
This issue occurred in the nxt_py_asgi_create_ip_address() function
where it tries to create an IP address / port number pair.
It does this by looking for the first ':' in the address and taking
everything after it as the port number. Like in the above error message,
if we tried to access the server @ 2001:db8:1:1:1ee7:dead:beef:cafe,
then we'd end up with the port number as 'db8:1:1:1ee7:dead:beef:cafe'.
There are two issues with this
1) The IP address and port number are already flowed through
separately.
2) Even if (1) wasn't true, it would still be broken for IPv6 as we'd
expect to a get an address literal like
[2001:db8:1:1:1ee7:dead:beef:cafe]:8080, however there was no code to
handle the []'s.
The fix is to simply not try looking for a port number. We pass a port
number into this function to use in the case where we don't find a port
number, we never will...
A further cleanup would be to flow through the server port number when
creating the 'server pair' PyTuple, rather than just using the hard
coded 80.
Closes: <https://github.com/nginx/unit/issues/793>
Closes: <https://github.com/nginx/unit/issues/874>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
This commit introduced the basic URI rewrite. It allows users to change request URI. Note the "rewrite" option ignores the contained query if any and the query from the request is preserverd.
An example:
"routes": [
{
"match": {
"uri": "/v1/test"
},
"action": {
"return": 200
}
},
{
"action": {
"rewrite": "/v1$uri",
"pass": "routes"
}
}
]
Reviewed-by: Alejandro Colomar <alx@nginx.com>
|
|
Normally Unit responds to HTTP requests by including a header like
Server: Unit/1.30.0
however it can sometimes be beneficial to withhold the version
information and in this case just respond with
Server: Unit
This patch adds a new "settings.http" boolean option called
server_version, which defaults to true, in which case the full version
information is sent. However this can be set to false, e.g
"settings": {
"http": {
"server_version": false
}
},
in which case Unit responds without the version information as the
latter example above shows.
Link: <https://www.ietf.org/rfc/rfc9110.html#section-10.2.4>
Closes: <https://github.com/nginx/unit/issues/158>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
