| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
Generic process-to-process shared memory exchange is no more required. Here,
it is transformed into a router-to-application pattern. The outgoing shared
memory segments collection is now the property of the application structure.
The applications connect to the router only, and the process only needs to group
the ports.
|
|
This is the port shared between all application processes which use it to pass
requests for processing. Using it significantly simplifies the request
processing code in the router. The drawback is 2 more file descriptors per each
configured application and more complex libunit message wait/read code.
|
|
The application process needs to request the shared memory segment from the
router instead of the latter pushing the segment before sending a request to
the application. This is required to simplify the communication between the
router and the application and to prepare the router for using the application
shared port and then the queue.
|
|
The application process needs to request the port from the router instead of the
latter pushing the port before sending a request to the application. This is
required to simplify the communication between the router and the application
and to prepare the router to use the application shared port and then the queue.
|
|
The goal is to minimize the number of (pid, id) to port hash lookups which
require a library mutex lock. The response port is found once per request,
while the read port is initialized at startup.
|
|
- Changed the port management callbacks to notifications, which e. g. avoids
the need to call the libunit function
- Added context and library instance reference counts for a safer resource
release
- Added the router main port initialization
|
|
This makes log format used in libunit consistent with the daemon, where milliseconds are printed only in the
debug log level.
Currently a compile time switch is used, since there's no support for runtime changing of a log level for now.
But in the future this should be a runtime condition, similar to nxt_log_time_handler().
|
|
The process abstraction has changed to:
setup(task, process)
start(task, process_data)
prefork(task, process, mp)
The prefork() occurs in the main process right before fork.
The file src/nxt_main_process.c is completely free of process
specific logic.
The creation of a process now supports a PROCESS_CREATED state. The
The setup() function of each process can set its state to either
created or ready. If created, a MSG_PROCESS_CREATED is sent to main
process, where external setup can be done (required for rootfs under
container).
The core processes (discovery, controller and router) doesn't need
external setup, then they all proceeds to their start() function
straight away.
In the case of applications, the load of the module happens at the
process setup() time and The module's init() function has changed
to be the start() of the process.
The module API has changed to:
setup(task, process, conf)
start(task, data)
As a direct benefit of the PROCESS_CREATED message, the clone(2) of
processes using pid namespaces now doesn't need to create a pipe
to make the child block until parent setup uid/gid mappings nor it
needs to receive the child pid.
|
|
An earlier attempt (ad6265786871) to resolve this condition on the
router's side added a new issue: the app could get a request before
acquiring a port.
|
|
Missing error log messages added.
|
|
Main process exiting before app process init may have caused hanging.
|
|
This is required for proper log file rotation action.
|
|
|
|
This closes #386 on GitHub.
|
|
Ruby and Java modules now use this function instead of own
implementations.
|
|
This issue was introduced in 2c7f79bf0a1f.
|
|
Uninitialized ctx_impl field may cause crash in application process.
To reproduce the issue, need to trigger shared memory buffer send error on
application side. In our case, send error caused by router process crash.
This issue was introduced in 2c7f79bf0a1f.
|
|
Found by Coverity (CID 353386).
|
|
- OOSM (out of shared memory). Sent by application process to router
when application reaches the limit of allocated shared memory and
needs more.
- SHM_ACK. Sent by router to application when the application's shared
memory is released and the OOSM flag is enabled for the segment.
This implements blocking mode (the library waits for SHM_ACK in case of
out of shared memory condition and retries allocating the required memory
amount) and non-blocking mode (the library notifies the application that
it's out of shared memory and returns control to the application module
that sets up the output queue and puts SHM_ACK in the main message loop).
|
|
|
|
The function unchains the buffer from the buffer's linked list.
|
|
Current shared memory buffer implementation uses fixed-size memory blocks,
allocating at least 16384 bytes. When application sends data in a large
number of small chunks, it makes sense to buffer them or use plain
memory buffers to improve performance and reduce memory footprint.
This patch introduces minimum size limit (1024 bytes) for shared
memory buffers.
|
|
Name and value in each header are 0-terminated, so additional 2 bytes
should be allocated for them. There were several attempts to add these
2 bytes to headers in language modules, but some modules weren't updated.
Also, adding these 2 bytes is specific to the implementation which may be
changed later, so extending this mechanics to modules may cause errors.
|
|
|
|
|
|
Found by Coverity (CID 349456).
|
|
Each request references the router process structure that owns all memory
maps. The process structure has a reference counter; each request increases
the counter to lock the structure in memory until request processing ends.
Incoming and outgoing buffers reference memory maps that the process owns,
so the process structure should be released only when all buffers are
released to avoid invalid memory access and a crash.
This describes the libunit library mechanism used for application processes.
The background of this issue is as follows:
The issue was found on buildbot when the router crashed during Java
websocket tests. The Java application receives a notification from the
master process; when the notification is processed, libunit deletes the
process structure from its process hash and decrements the use counter;
however, active websocket connections maintain their use counts on the
process structure. After that, when the master process is stopping the
application, libunit releases active websocket connections. At this point,
it's important to release the connections' memory buffers before the
corresponding process structure and all shared memory segments are released.
|
|
One alert per failed allocation is enough.
|
|
By design, Unit context is created for the thread which reads messages from
the router. However, Go request handlers are called in a separate goroutine
that may be executed in a different thread. To avoid a racing condition,
access to lists of free structures in the context should be serialized. This
patch should fix random crashes in Go applications under high load.
This is related to #253 and #309 issues on GitHub.
|
|
|
|
|
|
This patch contains various logging improvements and bugfixes found
during Java module development.
|
|
Previously, the nxt_router_prepare_msg() function expected server host among
other headers unmodified. It's not true anymore since normalization of the
Host header has been introduced in 77aad2c142a0.
The nxt_unit_split_host() function was removed. It didn't work correctly with
IPv6 literals. Anyway, after 77aad2c142a0 the port splitting is done in router
while Host header processing.
|
|
In case nxt_unit_tracking_read() failed, execution would jump to the error path,
where it could try to release buffers from uninitialized yet incoming_buf queue.
|
|
|
|
|
|
This issue was introduced in libunit commit (e0f0cd7d244a). All port
sockets in application should be in blocking mode whereas Unit itself
operates non-blocking sockets.
Having non-blocking sockets in application may cause send error during
intensive response packets generation.
See https://mailman.nginx.org/pipermail/unit/2018-October/000080.html.
|
|
|
|
Unit router process may send mmap file decritptor to the application process
for further information exchange. During this process there may be various
errors, which should be described in application error log. If file descriptor
cannot be properly transferred with 'new mmap' message, fd variable will
be assigned to -1 and further syscalls using this file descriptor will fail.
For 'new port' message fd is checked in the same way.
This commit adds early 'invalid file descriptor' diagnostic and write
corresponding message to error log.
Found by Coverity (CID 308515).
|
|
For the optimization purpose, function nxt_unit_remove_process() expects
lib->mutex to be locked. The function then moves ports queue into temporary
queue and releases mutex. In nxt_unit_done() there were two errors: mutex was
not locked before nxt_unit_remove_process() call and mutex was not destroyed.
It is hard to tell what was possible negative impact of this errors.
Found by Coverity (CID 308517).
|
|
Library now used in all language modules.
Old 'nxt_app_*' code removed.
See src/test/nxt_unit_app_test.c for usage sample.
|
