| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
If shared queue is empty, allocated read buffer should be explicitly
released.
Found by Coverity (CID 363943).
The issue was introduced in f5ba5973a0a3.
|
|
|
|
|
|
Removing unnecessary context operations from shared queue processing loop.
Initializing temporary queues only when required.
|
|
|
|
The issue occurred under highly concurrent request load in Go applications.
Such applications are multi-threaded but use a single libunit context; any
thread-safe code in the libunit context is only required for Go applications.
As a result of improper request state reset, the recycled request structure was
recovered in the released state, so further operations with this request
resulted in 'response already sent' warnings. However, the actual response was
never delivered to the router and the client.
|
|
The issue only occurred in Go applications because "port_send" is overloaded
only in Go. To reproduce it, send multiple concurrent requests to the
application after it has initialised. The warning message "[unit] [go] port
NNN:dd not found" is the first visible aspect of the issue; the second and more
valuable one is a closed connection, an error response, or a hanging response to
some requests.
When the application starts, it is unaware of the router's worker thread ports,
so it requests the ports from the router after receiving requests from the
corresponding router worker threads. When multiple requests are processed
simultaneously, the router port may be required by several requests, so request
processing starts only after the application receives the required port
information. The port should be added to the Go port repository after its
'ready' flag is updated. Otherwise, Unit may start processing some requests and
use the port before it is in the repository.
The issue was introduced in changeset 78836321a126.
|
|
Debug logging depends on macros defined in nxt_auto_config.h.
|
|
Application shared queue only capable to pass one shared memory buffer.
The rest buffers in chain needs to be send directly to application in response
to REQ_HEADERS_AC message.
The issue can be reproduced for configurations where 'body_buffer_size' is
greater than memory segment size (10 Mb). Requests with body size greater
than 10 Mb are just `stuck` e.g. not passed to application awaiting for more
data from router.
The bug was introduced in 1d84b9e4b459 (v1.19.0).
|
|
The socket is required for intercontextual communication in multithreaded apps.
|
|
|
|
The PORT_ACK message is the router's response to the application's NEW_PORT
message. After receiving PORT_ACK, the application is safe to process requests
using this port.
This message avoids a racing condition when the application starts processing a
request from the shared queue and sends REQ_HEADERS_ACK. The REQ_HEADERS_ACK
message contains the application port ID as reply_port, which the router uses
to send request data. When the application creates a new port, it
immediately sends it to the main router thread. Because the request is
processed outside the main thread, a racing condition can occur between the
receipt of the new port in the main thread and the receipt of REQ_HEADERS_ACK
in the worker router thread where the same port is specified as reply_port.
|
|
|
|
|
|
|
|
Until the mmap is received by the router, only the creator thread may use this
mmap, so the "mmap not found" state in the router is avoided.
|
|
Data in the queue and the socket are transmitted independently; special
READ_QUEUE and READ_SOCKET message types are used for synchronization.
The warning was accidentally committed with changeset 1d84b9e4b459.
|
|
|
|
Large-bodied requests are added to the request hash to be found when the body
arrives. However, changeset 1d84b9e4b459 introduced a bug: the 'in_hash' flag,
used to remove the request from the hash at request release, was cleared after
the first successful request lookup. As a result, the entry was never removed.
|
|
This change aids heap usage analysis in applications.
The alloc and free functions are also required for lvlhash due to the upcoming
threading support, because using main nxt_memalign() and nxt_free() isn't safe
in a multithreaded app environment. The reason is that these functions may use
thread-local structures which aren't initialized properly in applications.
|
|
|
|
Some of the pointers were not adjusted after frame's memory re-allocation.
Fortunately, this function was not used and the bug has no effect.
|
|
|
|
Two consecutive fd and fd2 fields replaced with array.
|
|
The default libunit behavior relies on blocking the recv() call for port file
descriptors, which an application may override if needed. For external
applications, port file descriptors were toggled to blocking mode before the
exec() call. If the exec() call failed, descriptor remained blocked, so the
process hanged while trying to read from it.
This patch moves file descriptor mode switch inside libunit.
|
|
|
|
The goal is to minimize the number of syscalls needed to deliver a message.
|
|
|
|
Generic process-to-process shared memory exchange is no more required. Here,
it is transformed into a router-to-application pattern. The outgoing shared
memory segments collection is now the property of the application structure.
The applications connect to the router only, and the process only needs to group
the ports.
|
|
This is the port shared between all application processes which use it to pass
requests for processing. Using it significantly simplifies the request
processing code in the router. The drawback is 2 more file descriptors per each
configured application and more complex libunit message wait/read code.
|
|
The application process needs to request the shared memory segment from the
router instead of the latter pushing the segment before sending a request to
the application. This is required to simplify the communication between the
router and the application and to prepare the router for using the application
shared port and then the queue.
|
|
The application process needs to request the port from the router instead of the
latter pushing the port before sending a request to the application. This is
required to simplify the communication between the router and the application
and to prepare the router to use the application shared port and then the queue.
|
|
The goal is to minimize the number of (pid, id) to port hash lookups which
require a library mutex lock. The response port is found once per request,
while the read port is initialized at startup.
|
|
- Changed the port management callbacks to notifications, which e. g. avoids
the need to call the libunit function
- Added context and library instance reference counts for a safer resource
release
- Added the router main port initialization
|
|
This makes log format used in libunit consistent with the daemon, where milliseconds are printed only in the
debug log level.
Currently a compile time switch is used, since there's no support for runtime changing of a log level for now.
But in the future this should be a runtime condition, similar to nxt_log_time_handler().
|
|
The process abstraction has changed to:
setup(task, process)
start(task, process_data)
prefork(task, process, mp)
The prefork() occurs in the main process right before fork.
The file src/nxt_main_process.c is completely free of process
specific logic.
The creation of a process now supports a PROCESS_CREATED state. The
The setup() function of each process can set its state to either
created or ready. If created, a MSG_PROCESS_CREATED is sent to main
process, where external setup can be done (required for rootfs under
container).
The core processes (discovery, controller and router) doesn't need
external setup, then they all proceeds to their start() function
straight away.
In the case of applications, the load of the module happens at the
process setup() time and The module's init() function has changed
to be the start() of the process.
The module API has changed to:
setup(task, process, conf)
start(task, data)
As a direct benefit of the PROCESS_CREATED message, the clone(2) of
processes using pid namespaces now doesn't need to create a pipe
to make the child block until parent setup uid/gid mappings nor it
needs to receive the child pid.
|
|
An earlier attempt (ad6265786871) to resolve this condition on the
router's side added a new issue: the app could get a request before
acquiring a port.
|
|
Missing error log messages added.
|
|
Main process exiting before app process init may have caused hanging.
|
|
This is required for proper log file rotation action.
|
|
|
|
This closes #386 on GitHub.
|
|
Ruby and Java modules now use this function instead of own
implementations.
|
|
This issue was introduced in 2c7f79bf0a1f.
|
|
Uninitialized ctx_impl field may cause crash in application process.
To reproduce the issue, need to trigger shared memory buffer send error on
application side. In our case, send error caused by router process crash.
This issue was introduced in 2c7f79bf0a1f.
|
|
Found by Coverity (CID 353386).
|
|
- OOSM (out of shared memory). Sent by application process to router
when application reaches the limit of allocated shared memory and
needs more.
- SHM_ACK. Sent by router to application when the application's shared
memory is released and the OOSM flag is enabled for the segment.
This implements blocking mode (the library waits for SHM_ACK in case of
out of shared memory condition and retries allocating the required memory
amount) and non-blocking mode (the library notifies the application that
it's out of shared memory and returns control to the application module
that sets up the output queue and puts SHM_ACK in the main message loop).
|
|
|
|
The function unchains the buffer from the buffer's linked list.
|
