| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Fixes: 1beab00ac ("Allow to set the HTTP response status")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The programs demonstrate handling requests with payloads larger than
4GiB which means they need to be written out to disk and so also
demonstrates the use of the file-system access mechanism.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
uwr_get_http_content_len() & uwr_get_http_total_content_sent() now
return a u64.
Two new functions were added
uwr_req_buf_copy()
Like uwr_req_buf_append() but just copies the data over what's already
there.
uwr_mem_splice_file()
This write(2)'s the request data directly from the shared memory to a
given file.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The types of some of the member of the luw_req structure increased to
64bits to allow for uploads larger than 4GiB.
Two new functions were added
luw_req_buf_copy()
Like luw_req_buf_append() but just copies the data over what's already
there.
luw_mem_splice_file()
This write(2)'s the request data directly from the shared memory to a
given file.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
libunit-wasm added two new functions, luw_req_buf_copy() &
luw_mem_splice_file(). See the previous two commits...
This second function takes a file-descriptor as one of its arguments, in
rusty we make this a Rust File object, then pass the underlying fd into
libunit-wasm.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is inspired by the likes of splice(2) and sendfile(2) in that it
takes data from one place and puts it in another.
This function write(2)'s the request data straight from the shared
memory to a given file (referenced by its file descriptor).
This is an alternative to using luw_req_buf_copy() and avoids an extra
copying of the request data.
E.g
/* In the request_handler */
if (total_bytes_wrote == 0) {
luw_init_ctx(&ctx, addr, 0);
luw_set_req_buf(&ctx, &request_buf, LUW_SRB_NONE);
fd = open("/var/tmp/large-file.dat", O_CREAT|O_TRUNC|O_WRONLY,
0666);
}
total_bytes_wrote += luw_mem_splice_file(addr, fd);
if (total_bytes_wrote == luw_get_http_content_len(&ctx)) {
close(fd);
total_bytes_wrote = 0;
luw_http_response_end();
}
NOTE:
We include a typedef definition for ssize_t in unit-wasm.h, to avoid
having a dependency on the wasi-sysroot when generating the rust
bindings.
ssize_t is defined in sys/types.h which is provided by libc and not the
compiler.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is analogous to luw_req_buf_append() but rather than appending
request data to the buffer it simply overwrites what's currently there.
This is needed to take advantage of the new ability to receive >4GiB
requests/payloads.
On a new request you would call luw_init_ctx(), luw_set_req_buf() &
open(2).
On subsequent calls to the request_handler (for this same HTTP
request/upload) you would call this new function and then write out the
data to a file.
E.g
/* In the request_handler */
if (total_bytes_wrote == 0) {
luw_init_ctx(&ctx, addr, 0);
luw_set_req_buf(&ctx, &request_buf, LUW_SRB_NONE);
fd = open("/var/tmp/large-file.dat", O_CREAT|O_TRUNC|O_WRONLY,
0666);
} else {
luw_req_buf_copy(&ctx, addr);
}
buf = luw_get_http_content(&ctx);
bytes_wrote = write(fd, buf, luw_get_http_content_sent(&ctx));
total_bytes_wrote += bytes_wrote;
if (total_bytes_wrote == luw_get_http_content_len(&ctx)) {
close(fd);
total_bytes_wrote = 0;
luw_http_response_end();
}
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The previous commit changed uwr_get_http_content_len() to return a u64
to allow for uploads larger than 4GiB, which now means this generates
compiler errors about type mismatches, expected usize got u64.
Cast the return value of uwr_get_http_content_len() to usize to match
that of TOTAL_RESPONSE_SENT.
(Making TOTAL_RESPONSE_SENT a u64 creates a larger trail of problems).
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Currently Wasm modules are limited to a 32bit address space (until at
least the memory64 work is completed). All the counters etc in the
request structure were u32's. Which matched with the 32bit memory
limitation.
However there is really no need to not allow >4GiB uploads that can be
saved off to disk or some such.
To do this we need to increase the ->content_len & ->total_content_sent
members to u64's and also adjust the return types of
(luw,uwr}_get_http_content_len() and
{luw,uwr}_get_http_total_content_sent() similarly.
However because we need the request structure to have the exact same
layout on 32bit (for Wasm modules) as it does on 64bit we need to re-jig
the order of some of these members and add a four-byte padding member.
Thus the request structure now looks like on 32bit (as shown by
pahole(1))
struct luw_req {
u32 method_off; /* 0 4 */
u32 method_len; /* 4 4 */
u32 version_off; /* 8 4 */
u32 version_len; /* 12 4 */
u32 path_off; /* 16 4 */
u32 path_len; /* 20 4 */
u32 query_off; /* 24 4 */
u32 query_len; /* 28 4 */
u32 remote_off; /* 32 4 */
u32 remote_len; /* 36 4 */
u32 local_addr_off; /* 40 4 */
u32 local_addr_len; /* 44 4 */
u32 local_port_off; /* 48 4 */
u32 local_port_len; /* 52 4 */
u32 server_name_off; /* 56 4 */
u32 server_name_len; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
u64 content_len; /* 64 8 */
u64 total_content_sent; /* 72 8 */
u32 content_sent; /* 80 4 */
u32 content_off; /* 84 4 */
u32 request_size; /* 88 4 */
u32 nr_fields; /* 92 4 */
u32 tls; /* 96 4 */
char __pad[4]; /* 100 4 */
struct luw_hdr_field fields[]; /* 104 0 */
/* size: 104, cachelines: 2, members: 25 */
/* last cacheline: 40 bytes */
};
and the same structure (taken from Unit) compiled as 64bit
struct nxt_wasm_request_s {
uint32_t method_off; /* 0 4 */
uint32_t method_len; /* 4 4 */
uint32_t version_off; /* 8 4 */
uint32_t version_len; /* 12 4 */
uint32_t path_off; /* 16 4 */
uint32_t path_len; /* 20 4 */
uint32_t query_off; /* 24 4 */
uint32_t query_len; /* 28 4 */
uint32_t remote_off; /* 32 4 */
uint32_t remote_len; /* 36 4 */
uint32_t local_addr_off; /* 40 4 */
uint32_t local_addr_len; /* 44 4 */
uint32_t local_port_off; /* 48 4 */
uint32_t local_port_len; /* 52 4 */
uint32_t server_name_off; /* 56 4 */
uint32_t server_name_len; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
uint64_t content_len; /* 64 8 */
uint64_t total_content_sent; /* 72 8 */
uint32_t content_sent; /* 80 4 */
uint32_t content_off; /* 84 4 */
uint32_t request_size; /* 88 4 */
uint32_t nfields; /* 92 4 */
uint32_t tls; /* 96 4 */
char __pad[4]; /* 100 4 */
nxt_wasm_http_field_t fields[]; /* 104 0 */
/* size: 104, cachelines: 2, members: 25 */
/* last cacheline: 40 bytes */
};
We can see the structures have the same layout, same size and no
padding.
We need the __pad member as otherwise I saw gcc and clang on Alpine
Linux automatically add the 'packed' attribute to the structure which
made the two structures not match.
Link: <https://github.com/WebAssembly/memory64>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The new uwr_http_set_response_status() function allows to set the HTTP
response status in Rust WebAssembly modules. It takes one of the
luw_http_status_t response status values.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This adds a new luw_http_set_response_status() function that takes one
of the luw_http_status_t response status codes.
This function should be called before any calls to
luw_http_send_headers() or luw_http_send_response().
This function calls into Unit via a new function import,
nxt_wasm_set_resp_status().
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Change the minimum clang/llvm version to 9.0 (as noted here[0]).
Provide an alternate command for installing the libclang wasm runtime
library.
[0]: https://github.com/WebAssembly/wasi-sdk/blob/main/README.md?plain=1#L19
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This requires some nefarious use of rustup and the funky rustup command
line comes courtesy of <https://dentrassi.de/2020/06/17/headless-installation-of-cargo-and-rust/>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Must of got out of whack when editing it from the C version.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Previously bindgen was picking the 'constified_enum'. E.g it would turn
the luw_srb_flags_t enum
typedef enum {
LUW_SRB_NONE = 0x00,
LUW_SRB_APPEND = 0x01,
LUW_SRB_ALLOC = 0x02,
LUW_SRB_FULL_SIZE = 0x04,
LUW_SRB_FLAGS_ALL = (LUW_SRB_NONE|LUW_SRB_APPEND|LUW_SRB_ALLOC|
LUW_SRB_FULL_SIZE)
} luw_srb_flags_t;
into
pub const luw_srb_flags_t_LUW_SRB_NONE: luw_srb_flags_t = 0;
pub const luw_srb_flags_t_LUW_SRB_APPEND: luw_srb_flags_t = 1;
pub const luw_srb_flags_t_LUW_SRB_ALLOC: luw_srb_flags_t = 2;
pub const luw_srb_flags_t_LUW_SRB_FULL_SIZE: luw_srb_flags_t = 4;
pub const luw_srb_flags_t_LUW_SRB_FLAGS_ALL: luw_srb_flags_t = 7;
But then this requires some further changes to make the names nicer
without the type prefixed.
This will only be exasperated when adding an enum containing the HTTP
status codes...
So instead, tell bindgen to use the 'rustified_enum' method which
produces this
pub enum luw_srb_flags_t {
LUW_SRB_NONE = 0,
LUW_SRB_APPEND = 1,
LUW_SRB_ALLOC = 2,
LUW_SRB_FULL_SIZE = 4,
LUW_SRB_FLAGS_ALL = 7,
}
which in theory requires no extra changes (it doesn't with the http
status codes enum), however in this specific case, because these are
actually bitflags we still need to cast them to u32 so they can be OR'd.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
There is no need in this case to declare REQUEST_BUF as a global
variable. Declaring it local to uwr_request_handler() lets us get rid of
the unsafe code blocks.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Update the echo-request and upload-reflector examples for the new
uwr_http_add_header_content_type() and uwr_http_add_header_content_len()
functions.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When the uwr_get_http_content_str() function, which returns the request
body content as a string, was added it used uwr_get_http_content_len()
to determine the length of the returned string (the request body content
is not null-terminated).
This could potentially in some circumstances return too much data if the
request was split over multiple calls into the Wasm module and
uwr_get_http_content_str() was called before all the data had actually
been received.
Instead use the newly introduced uwr_get_http_total_content_sent()
function to determine the amount of data to return in the string as it
currently stands.
Fixes: bf968c9 ("Rust/rusty: Add uwr_get_http_content_str()")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This returns the total amount of content that the Wasm module has
received so far.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This may not matter in rust (if you use a function before the compiler
has seen a definition for it), but anyway in preparation for adding a
uwr_get_http_total_content_sent() function, put the content length
related functions before the functions that return the content.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This function returns the total amount of content that the Wasm module
has received so far.
This will be used in the rusty API's uwr_get_http_content_str() function
which returns the body content as a string. Due to the body content not
being null-terminated we need to know how much data to use for the
string, for which we are currently using uwr_get_http_content_len(),
which could in some cases return too much if a request is being split
over multiple calls the module.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
When I renamed it from minimal to hello-world, it stopped being built
due to the make target name being the same as the directory name
(hello-world).
Rename the make target to rust-hello-world which also matches the naming
of the rest of the targets.
Fixes: 656c036 ("examples/rust: Add a minimal hello world rust example")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is about the smallest it can be.
Its Unit application config would look like
"applications": {
"rust-hello-world": {
"type": "wasm",
"module": "/path/to/unit-wasm/examples/rust/hello-world/target/wasm32-wasi/debug/rust_hello_world.wasm",
"request_handler": "uwr_request_handler",
"malloc_handler": "luw_malloc_handler",
"free_handler": "luw_free_handler"
}
}
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This adds the following convenience functions for adding HTTP response
headers, Content-Type & Content-length
uwr_http_add_header_content_type(ctx: *mut luw_ctx_t, ctype: &str);
uwr_http_add_header_content_len(ctx: *mut luw_ctx_t);
These are perhaps the two most common headers so it makes sense to
reduce the effort to adding them.
E.g before
uwr_http_add_header(&ctx, "Content-Type", "text/plain");
uwr_http_add_header(
ctx,
"Content-Length",
&format!("{}", uwr_get_response_data_size(ctx)),
);
after
uwr_http_add_header_content_type(ctx, "text/plain");
uwr_http_add_header_content_len(ctx);
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This function is like uwr_get_http_content() except that it returns a
Rust str.
This is more convenient if you want to operate on the body content
within Rust.
It's worth noting that uwr_get_http_content() returns a non null
terminated buffer which makes it tricky to work with in Rust.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Use uwr_get_response_data_size() instead of uwr_get_http_content_len()
for the Content-Length, this is more appropriate as this will give the
length of the response data rather than the request data.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
luw_http_add_header() no longer takes an idx argument.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
luw_http_add_header() no longer takes an idx argument.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
luw_http_add_header() no longer takes an idx argument.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This was done by 'cp API-C.md API-Rust.md' and then adjusted as
necessary.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
The various uwr_get_ functions should take the context pointer as a
const.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Technically the context pointer can be passed in const even though we
then un-const it passing it to the luw_foreach_http_hdr() macro.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This was used to specify the index of the response header being added,
starting at 0 and incrementing by one for each header.
Instead of having the programmer specify this, track it internally.
We add an extra check in luw_http_add_header() to make sure we aren't
trying to add more headers than we said with luw_http_init_headers(), if
we are, simply return.
This updates the API-C.md and the various examples and 'rusty' API
wrapper.
Suggested-by: Liam Crilly <liam@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This reverts commit 011c3ba3f7bc466a04101f81d4f6186001b7aad4.
This was committed in error...
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
|
|
This adds a new 'Setup a Suitable Environment' section that replaces the
'Getting Started' section and it's subsections.
This provides a more comprehensive guide to setting up a suitable
environment on some specific systems (currently Fedora/CentOS/RHEL etc
& Debian/Ubuntu).
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This now varies if your using Fedora or FreeBSD and have the right
package installed.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
Rather than using and rpm(8) and grep(1) to find the location to store
the libclang_rt.builtins-wasm32.a file use clangs -print-runtime-dir
option and strip the last component.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
|
This is for the new 'rusty' API.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
