diff options
Diffstat (limited to '')
202 files changed, 4743 insertions, 6555 deletions
@@ -67,4 +67,6 @@ ea073fb3cb75abfb4be5dc12402de73e0c20da60 1.28.0 16e01c5fead470d8f630613b973a50a33fd0d0f2 1.28.0-1 37cac7fec92e5656d8a03a8594ade131c3391f45 1.29.0 2927b72847d44913233fa303438a14ec313d860e 1.29.0-1 +fa0227b7f62691a186d752ace475868de49e9fce 1.29.1 e7b7f2bb04e8c6f4cbe6374fd6960d4465654215 1.29.1-1 +2692a5823c403a4e209681943e32a4907317d14b 1.30.0 @@ -1,4 +1,32 @@ +Changes with Unit 1.30.0 10 May 2023 + + *) Change: remove Unix domain listen sockets upon reconfiguration. + + *) Feature: basic URI rewrite support. + + *) Feature: NJS loadable modules support. + + *) Feature: per-application logging. + + *) Feature: conditional logging of route selection. + + *) Feature: support the keys API on the request objects in NJS. + + *) Feature: default values for 'make install' pathnames such as prefix; + this allows to './configure && make && sudo make install'. + + *) Feature: "server_version" setting to omit the version token from + "Server" header field. + + *) Bugfix: request header field values could be corrupted in some cases; + the bug had appeared in 1.29.0. + + *) Bugfix: PHP error handling (added missing 403 and 404 errors). + + *) Bugfix: Perl applications crash on second responder call. + + Changes with Unit 1.29.1 28 Feb 2023 *) Bugfix: stop creating world-writeable directories. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..aea287f6 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,74 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to making participation in our project and +our community a harassment-free experience for everyone, regardless of age, body +size, disability, ethnicity, sex characteristics, gender identity and expression, +level of experience, education, socio-economic status, nationality, personal +appearance, race, religion, or sexual identity and orientation. + +## Our Standards + +Examples of behavior that contributes to creating a positive environment +include: + +- Using welcoming and inclusive language +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members + +Examples of unacceptable behavior by participants include: + +- The use of sexualized language or imagery and unwelcome sexual attention or + advances +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or electronic + address, without explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Our Responsibilities + +Project maintainers are responsible for clarifying the standards of acceptable +behavior and are expected to take appropriate and fair corrective action in +response to any instances of unacceptable behavior. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct, or to ban temporarily or +permanently any contributor for other behaviors that they deem inappropriate, +threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies both within project spaces and in public spaces +when an individual is representing the project or its community. Examples of +representing a project or community include using an official project e-mail +address, posting via an official social media account, or acting as an appointed +representative at an online or offline event. Representation of a project may be +further defined and clarified by project maintainers. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by contacting the moderation team at nginx-oss-community@f5.com. All +complaints will be reviewed and investigated and will result in a response that +is deemed necessary and appropriate to the circumstances. The project team is +obligated to maintain confidentiality with regard to the reporter of an incident. +Further details of specific enforcement policies may be posted separately. + +Project maintainers who do not follow or enforce the Code of Conduct in good +faith may face temporary or permanent repercussions as determined by other +members of the project's leadership. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 1.4, +available at <https://www.contributor-covenant.org/version/1/4/code-of-conduct.html> + +For answers to common questions about this code of conduct, see +<https://www.contributor-covenant.org/faq> @@ -5,19 +5,17 @@  NGINX Unit is a lightweight and versatile open-source server that has -three core capabilities: +two primary capabilities: -- it is an HTTP reverse proxy, -- a web server for static media assets, -- and an application server that runs code in seven languages. +- serves static media assets, +- runs application code in seven languages. -We are building a universal tool that compresses several layers of the modern -application stack into a potent, coherent solution with a focus on performance, -low latency, and scalability. It is intended as a building block for any web -architecture regardless of its complexity, from enterprise-scale deployments to -your pet's homepage. +Unit compresses several layers of the modern application stack into a potent, +coherent solution with a focus on performance, low latency, and scalability. It +is intended as a universal building block for any web architecture regardless +of its complexity, from enterprise-scale deployments to your pet's homepage. -Unit's native RESTful JSON API enables dynamic updates with zero interruptions +Its native RESTful JSON API enables dynamic updates with zero interruptions and flexible configuration, while its out-of-the-box productivity reliably scales to production-grade workloads. We achieve that with a complex, asynchronous, multithreading architecture comprising multiple processes to @@ -162,7 +160,7 @@ For full details of configuration management, see the ## Community - The go-to place to start asking questions and share your thoughts is - our [Slack channel](https://nginxcommunity.slack.com/). + our [Slack channel](https://community.nginx.org/joinslack). - Our [GitHub issues page](https://github.com/nginx/unit/issues) offers space for a more technical discussion at your own pace. diff --git a/auto/echo/build b/auto/echo/build index d66c6951..b2396cd0 100644 --- a/auto/echo/build +++ b/auto/echo/build @@ -5,15 +5,15 @@ $echo 'building an "echo" program' -rm -f $NXT_BUILD_DIR/echo +rm -f $NXT_BUILD_DIR/bin/echo -nxt_echo_test="$CC -o $NXT_BUILD_DIR/echo -O $NXT_CC_OPT +nxt_echo_test="$CC -o $NXT_BUILD_DIR/bin/echo -O $NXT_CC_OPT auto/echo/echo.c $NXT_LD_OPT" # "|| true" is to bypass "set -e" setting. nxt_echo_err=`$nxt_echo_test 2>&1 || true` -if [ ! -x $NXT_BUILD_DIR/echo ]; then +if [ ! -x $NXT_BUILD_DIR/bin/echo ]; then $echo $echo $0: error: cannot build an \"echo\" program: $echo @@ -24,4 +24,4 @@ if [ ! -x $NXT_BUILD_DIR/echo ]; then exit 1 fi -echo=$NXT_BUILD_DIR/echo +echo=$NXT_BUILD_DIR/bin/echo @@ -11,23 +11,31 @@ cat << END --cc-opt=OPTIONS set additional C compiler options --ld-opt=OPTIONS set additional linker options - --prefix=DIRECTORY set prefix for relative pathnames, default: none - --bindir=DIRECTORY set user executables directory name - default: "$NXT_BINDIR" - --sbindir=DIRECTORY set system admin executables directory name - default: "$NXT_SBINDIR" - --libdir=DIRECTORY set library directory name, default: "$NXT_LIBDIR" - --incdir=DIRECTORY set includes directory name, default: "$NXT_INCDIR" - --mandir=DIRECTORY set man pages directory name, default: "$NXT_MANDIR" - --modules=DIRECTORY set modules directory name, default: "$NXT_MODULES" - --state=DIRECTORY set state directory name, default: "$NXT_STATE" - --tmp=DIRECTORY set tmp directory name, default: "$NXT_TMP" - - --pid=FILE set pid filename, default: "$NXT_PID" - --log=FILE set log filename, default: "$NXT_LOG" + --prefix=DIR default: "/usr/local" + --exec-prefix=DIR default: "\$prefix" + --bindir=DIR default: "\$exec_prefix/bin" + --sbindir=DIR default: "\$exec_prefix/sbin" + --includedir=DIR default: "\$prefix/include" + --libdir=DIR default: "\$exec_prefix/lib" + --modulesdir=DIR default: "\$libdir/unit/modules" + --datarootdir=DIR default: "\$prefix/share" + --mandir=DIR default: "\$datarootdir/man" + --localstatedir=DIR default: "\$prefix/var" + --statedir=DIR default: "\$localstatedir/lib/unit" + --runstatedir=DIR default: "\$localstatedir/run/unit" + --logdir=DIR default: "\$localstatedir/log/unit" + --tmpdir=DIR default: "/tmp" + + --incdir=DIR [deprecated] synonym for --includedir + --modules=DIR [deprecated] synonym for --modulesdir + --state=DIR [deprecated] synonym for --statedir + --tmp=DIR [deprecated] synonym for --tmpdir + + --pid=FILE set pid filename, default: "\$runstatedir/unit.pid" + --log=FILE set log filename, default: "\$logdir/unit.log" --control=ADDRESS set address of control API socket - default: "$NXT_CONTROL" + default: "unix:\$runstatedir/control.unit.sock" --user=USER set non-privileged processes to run as specified user default: "$NXT_USER" @@ -7,9 +7,6 @@ $echo "creating $NXT_MAKEFILE" -mkdir -p $NXT_BUILD_DIR/src \ - $NXT_BUILD_DIR/src/test - cat << END > $NXT_MAKEFILE @@ -25,8 +22,8 @@ NXT_MODULE_LINK = $NXT_MODULE_LINK all: $NXT_DAEMON manpage .PHONY: $NXT_DAEMON manpage -$NXT_DAEMON: $NXT_BUILD_DIR/$NXT_DAEMON -manpage: $NXT_BUILD_DIR/unitd.8 +$NXT_DAEMON: $NXT_BUILD_DIR/sbin/$NXT_DAEMON +manpage: $NXT_BUILD_DIR/share/man/man8/unitd.8 END @@ -35,7 +32,7 @@ END $echo -n "NXT_LIB_INCS =" >> $NXT_MAKEFILE -for nxt_inc in src $NXT_BUILD_DIR +for nxt_inc in src $NXT_BUILD_DIR/include do $echo -n " -I $nxt_inc" >> $NXT_MAKEFILE done @@ -88,20 +85,17 @@ END cat << END >> $NXT_MAKEFILE -libnxt: $NXT_BUILD_DIR/$NXT_LIB_SHARED $NXT_BUILD_DIR/$NXT_LIB_STATIC +libnxt: $NXT_BUILD_DIR/lib/$NXT_LIB_SHARED $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC -$NXT_BUILD_DIR/$NXT_LIB_SHARED: \$(NXT_LIB_OBJS) - \$(NXT_SHARED_LOCAL_LINK) -o $NXT_BUILD_DIR/$NXT_LIB_SHARED \\ - \$(NXT_LIB_OBJS) \\ +$NXT_BUILD_DIR/lib/$NXT_LIB_SHARED: \$(NXT_LIB_OBJS) + \$(NXT_SHARED_LOCAL_LINK) -o \$@ \$(NXT_LIB_OBJS) \\ $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS -$NXT_BUILD_DIR/$NXT_LIB_STATIC: \$(NXT_LIB_OBJS) - $NXT_STATIC_LINK $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ - \$(NXT_LIB_OBJS) +$NXT_BUILD_DIR/lib/$NXT_LIB_STATIC: \$(NXT_LIB_OBJS) + $NXT_STATIC_LINK \$@ \$(NXT_LIB_OBJS) -$NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC: \$(NXT_LIB_UNIT_OBJS) - $NXT_STATIC_LINK $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC \\ - \$(NXT_LIB_UNIT_OBJS) +$NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC: \$(NXT_LIB_UNIT_OBJS) + $NXT_STATIC_LINK \$@ \$(NXT_LIB_UNIT_OBJS) END @@ -195,55 +189,55 @@ tests: $NXT_BUILD_DIR/tests $NXT_BUILD_DIR/utf8_file_name_test \\ $NXT_BUILD_DIR/unit_websocket_echo $NXT_BUILD_DIR/tests: \$(NXT_TEST_OBJS) \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/tests \\ \$(CFLAGS) \$(NXT_TEST_OBJS) \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/utf8_file_name_test: $NXT_LIB_UTF8_FILE_NAME_TEST_SRCS \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \$(CC) \$(CFLAGS) \$(NXT_LIB_INCS) $NXT_LIB_AUX_CFLAGS \\ -o $NXT_BUILD_DIR/utf8_file_name_test \\ $NXT_LIB_UTF8_FILE_NAME_TEST_SRCS \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/ncq_test: $NXT_BUILD_DIR/src/test/nxt_ncq_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/ncq_test \\ \$(CFLAGS) $NXT_BUILD_DIR/src/test/nxt_ncq_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/vbcq_test: $NXT_BUILD_DIR/src/test/nxt_vbcq_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/vbcq_test \\ \$(CFLAGS) $NXT_BUILD_DIR/src/test/nxt_vbcq_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/unit_app_test: $NXT_BUILD_DIR/src/test/nxt_unit_app_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/unit_app_test \\ \$(CFLAGS) $NXT_BUILD_DIR/src/test/nxt_unit_app_test.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/unit_websocket_chat: \\ $NXT_BUILD_DIR/src/test/nxt_unit_websocket_chat.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/unit_websocket_chat \\ \$(CFLAGS) $NXT_BUILD_DIR/src/test/nxt_unit_websocket_chat.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS $NXT_BUILD_DIR/unit_websocket_echo: \\ $NXT_BUILD_DIR/src/test/nxt_unit_websocket_echo.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/unit_websocket_echo \\ \$(CFLAGS) $NXT_BUILD_DIR/src/test/nxt_unit_websocket_echo.o \\ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC \\ + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \\ $NXT_LD_OPT $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS END @@ -264,7 +258,7 @@ END fi -NXT_MAKE_INCS="src $NXT_BUILD_DIR" +NXT_MAKE_INCS="src $NXT_BUILD_DIR/include" NXT_MAKE_SRCS="$NXT_SRCS" @@ -298,10 +292,10 @@ $echo >> $NXT_MAKEFILE cat << END >> $NXT_MAKEFILE -$NXT_BUILD_DIR/$NXT_DAEMON: $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ +$NXT_BUILD_DIR/sbin/$NXT_DAEMON: $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ \$(NXT_OBJS) - \$(NXT_EXEC_LINK) -o $NXT_BUILD_DIR/$NXT_DAEMON \$(CFLAGS) \\ - \$(NXT_OBJS) $NXT_BUILD_DIR/$NXT_LIB_STATIC \\ + \$(NXT_EXEC_LINK) -o \$@ \$(CFLAGS) \\ + \$(NXT_OBJS) $NXT_BUILD_DIR/lib/$NXT_LIB_STATIC \\ $NXT_LIBM $NXT_LIBS $NXT_LIB_AUX_LIBS END @@ -311,11 +305,12 @@ END cat << END >> $NXT_MAKEFILE -$NXT_BUILD_DIR/unitd.8: docs/man/unitd.8.in $NXT_BUILD_DIR/nxt_auto_config.h +$NXT_BUILD_DIR/share/man/man8/unitd.8: docs/man/man8/unitd.8.in \\ + $NXT_BUILD_DIR/include/nxt_auto_config.h sed -e "s|%%ERROR_LOG_PATH%%|$NXT_LOG|" \\ -e "s|%%PID_PATH%%|$NXT_PID|" \\ -e "s|%%SOCKET_PATH%%|$NXT_CONTROL|" \\ - < docs/man/unitd.8.in > \$@ + < docs/man/man8/unitd.8.in > \$@ END @@ -365,14 +360,19 @@ install-check: ${NXT_DAEMON}-install: $NXT_DAEMON install-check test -d \$(DESTDIR)$NXT_SBINDIR \ || install -d \$(DESTDIR)$NXT_SBINDIR - install -p $NXT_BUILD_DIR/$NXT_DAEMON \$(DESTDIR)$NXT_SBINDIR/ - test -d \$(DESTDIR)$NXT_STATE \ - || install -d \$(DESTDIR)$NXT_STATE + install -p $NXT_BUILD_DIR/sbin/$NXT_DAEMON \$(DESTDIR)$NXT_SBINDIR/ + test -d \$(DESTDIR)$NXT_STATEDIR \ + || install -d \$(DESTDIR)$NXT_STATEDIR + test -d \$(DESTDIR)$NXT_LOGDIR \ + || install -d \$(DESTDIR)$NXT_LOGDIR + test -d \$(DESTDIR)$NXT_RUNSTATEDIR \ + || install -d \$(DESTDIR)$NXT_RUNSTATEDIR manpage-install: manpage install-check test -d \$(DESTDIR)$NXT_MANDIR/man8 \ || install -d \$(DESTDIR)$NXT_MANDIR/man8 - install -p -m644 $NXT_BUILD_DIR/unitd.8 \$(DESTDIR)$NXT_MANDIR/man8/ + install -p -m644 $NXT_BUILD_DIR/share/man/man8/unitd.8 \ + \$(DESTDIR)$NXT_MANDIR/man8/ .PHONY: uninstall ${NXT_DAEMON}-uninstall manpage-uninstall @@ -392,13 +392,13 @@ cat << END >> $NXT_MAKEFILE .PHONY: libunit-install libunit-uninstall -libunit-install: $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC +libunit-install: $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC test -d \$(DESTDIR)$NXT_LIBDIR \ || install -d \$(DESTDIR)$NXT_LIBDIR - install -p -m u=rw,go=r $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC \ + install -p -m u=rw,go=r $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC \ \$(DESTDIR)$NXT_LIBDIR/ - test -d \$(DESTDIR)$NXT_INCDIR \ - || install -d \$(DESTDIR)$NXT_INCDIR + test -d \$(DESTDIR)$NXT_INCLUDEDIR \ + || install -d \$(DESTDIR)$NXT_INCLUDEDIR install -p -m u=rw,go=r src/nxt_unit.h \ src/nxt_unit_field.h \ src/nxt_unit_request.h \ @@ -406,25 +406,25 @@ libunit-install: $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC src/nxt_unit_sptr.h \ src/nxt_unit_typedefs.h \ src/nxt_unit_websocket.h \ - $NXT_BUILD_DIR/nxt_auto_config.h \ - $NXT_BUILD_DIR/nxt_version.h \ + $NXT_BUILD_DIR/include/nxt_auto_config.h \ + $NXT_BUILD_DIR/include/nxt_version.h \ src/nxt_websocket_header.h \ - \$(DESTDIR)$NXT_INCDIR/ + \$(DESTDIR)$NXT_INCLUDEDIR/ libunit-uninstall: rm -f \$(DESTDIR)$NXT_LIBDIR/$NXT_LIB_UNIT_STATIC @rmdir -p \$(DESTDIR)$NXT_LIBDIR 2>/dev/null || true - rm -f \$(DESTDIR)$NXT_INCDIR/nxt_unit.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_field.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_request.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_response.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_sptr.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_typedefs.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_unit_websocket.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_auto_config.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_version.h \ - \$(DESTDIR)$NXT_INCDIR/nxt_websocket_header.h - @rmdir -p \$(DESTDIR)$NXT_INCDIR 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_field.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_request.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_response.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_sptr.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_typedefs.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_unit_websocket.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_auto_config.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_version.h \ + \$(DESTDIR)$NXT_INCLUDEDIR/nxt_websocket_header.h + @rmdir -p \$(DESTDIR)$NXT_INCLUDEDIR 2>/dev/null || true END diff --git a/auto/modules/go b/auto/modules/go index a8596bf3..86dfb62d 100644 --- a/auto/modules/go +++ b/auto/modules/go @@ -119,15 +119,15 @@ ${NXT_GO}-install-src: install -p -m644 ./go/* \$(DESTDIR)\$(NXT_GO_DST)/src/${NXT_GO_PKG}/ ${NXT_GO}-install-env: \$(DESTDIR)\$(NXT_GO_DST)/src/${NXT_GO_PKG}/env.go \ - ${NXT_VERSION_H} ${NXT_BUILD_DIR}/${NXT_LIB_UNIT_STATIC} + ${NXT_VERSION_H} ${NXT_BUILD_DIR}/lib/${NXT_LIB_UNIT_STATIC} \$(DESTDIR)\$(NXT_GO_DST)/src/${NXT_GO_PKG}/env.go: install -d \$(DESTDIR)\$(NXT_GO_DST)/src/${NXT_GO_PKG} $echo "package unit" > \$@ $echo "/*" >> \$@ $echo "#cgo CFLAGS: ${CFLAGS} ${NXT_CC_OPT}" >> \$@ - $echo "#cgo CPPFLAGS: -I${PWD}/src -I${PWD}/${NXT_BUILD_DIR}" >> \$@ - $echo "#cgo LDFLAGS: -L${PWD}/${NXT_BUILD_DIR} ${NXT_GO_LDFLAGS} ${NXT_LD_OPT}" >> \$@ + $echo "#cgo CPPFLAGS: -I${PWD}/src -I${PWD}/${NXT_BUILD_DIR}/include" >> \$@ + $echo "#cgo LDFLAGS: -L${PWD}/${NXT_BUILD_DIR}/lib ${NXT_GO_LDFLAGS} ${NXT_LD_OPT}" >> \$@ $echo "*/" >> \$@ $echo 'import "C"' >> \$@ diff --git a/auto/modules/java b/auto/modules/java index bdf17022..d87f93c5 100644 --- a/auto/modules/java +++ b/auto/modules/java @@ -63,7 +63,7 @@ fi . $NXT_AUTOCONF_DATA -NXT_JARS=${NXT_JARS=$NXT_MODULES} +NXT_JARS=${NXT_JARS=$NXT_MODULESDIR} NXT_JAVA_MODULE=${NXT_JAVA_MODULE=java} NXT_JAVA_LIB_PATH=${NXT_JAVA_LIB_PATH=} @@ -320,7 +320,7 @@ fi NXT_JAVA_MOUNTS_HEADER=nxt_${NXT_JAVA_MODULE}_mounts.h -cat << END > $NXT_BUILD_DIR/$NXT_JAVA_MOUNTS_HEADER +cat << END > $NXT_BUILD_DIR/include/$NXT_JAVA_MOUNTS_HEADER #ifndef _NXT_JAVA_MOUNTS_H_INCLUDED_ #define _NXT_JAVA_MOUNTS_H_INCLUDED_ @@ -520,24 +520,24 @@ cat << END >> $NXT_MAKEFILE all: ${NXT_JAVA_MODULE} -${NXT_JAVA_MODULE}: $NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so \ +${NXT_JAVA_MODULE}: $NXT_BUILD_DIR/lib/unit/modules/${NXT_JAVA_MODULE}.unit.so \ $NXT_BUILD_DIR/$NXT_UNIT_JAR \ $NXT_BUILD_DIR/$NXT_WS_API_JAR -$NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so: $nxt_objs - \$(NXT_MODULE_LINK) -o $NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so \\ - $nxt_objs $NXT_JAVA_LDFLAGS $NXT_LD_OPT +$NXT_BUILD_DIR/lib/unit/modules/${NXT_JAVA_MODULE}.unit.so: $nxt_objs + \$(NXT_MODULE_LINK) -o \$@ $nxt_objs $NXT_JAVA_LDFLAGS $NXT_LD_OPT install: ${NXT_JAVA_MODULE}-install -${NXT_JAVA_MODULE}-install: $NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so \\ +${NXT_JAVA_MODULE}-install: \\ + $NXT_BUILD_DIR/lib/unit/modules/${NXT_JAVA_MODULE}.unit.so \\ $NXT_BUILD_DIR/$NXT_UNIT_JAR \\ $NXT_BUILD_DIR/$NXT_WS_API_JAR \\ java-shared-install - install -d \$(DESTDIR)$NXT_MODULES - install -p $NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so \\ - \$(DESTDIR)$NXT_MODULES/ + install -d \$(DESTDIR)$NXT_MODULESDIR + install -p $NXT_BUILD_DIR/lib/unit/modules/${NXT_JAVA_MODULE}.unit.so \\ + \$(DESTDIR)$NXT_MODULESDIR/ install -d \$(DESTDIR)$NXT_JARS install -p -m 0644 $NXT_BUILD_DIR/$NXT_UNIT_JAR \$(DESTDIR)$NXT_JARS/ install -p -m 0644 $NXT_BUILD_DIR/$NXT_WS_API_JAR \$(DESTDIR)$NXT_JARS/ @@ -546,8 +546,8 @@ ${NXT_JAVA_MODULE}-install: $NXT_BUILD_DIR/${NXT_JAVA_MODULE}.unit.so \\ uninstall: ${NXT_JAVA_MODULE}-uninstall ${NXT_JAVA_MODULE}-uninstall: java-shared-uninstall - rm -f \$(DESTDIR)$NXT_MODULES/${NXT_JAVA_MODULE}.unit.so - @rmdir -p \$(DESTDIR)$NXT_MODULES 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_MODULESDIR/${NXT_JAVA_MODULE}.unit.so + @rmdir -p \$(DESTDIR)$NXT_MODULESDIR 2>/dev/null || true rm -f \$(DESTDIR)$NXT_JARS/$NXT_UNIT_JAR rm -f \$(DESTDIR)$NXT_JARS/$NXT_WS_API_JAR @rmdir -p \$(DESTDIR)$NXT_JARS 2>/dev/null || true diff --git a/auto/modules/nodejs b/auto/modules/nodejs index 7d4f8581..968f3fdf 100644 --- a/auto/modules/nodejs +++ b/auto/modules/nodejs @@ -129,7 +129,7 @@ NXT_NODE_VERSION_FILE=${NXT_BUILD_DIR}/src/${NXT_NODE}/version.h NXT_NODE_PACKAGE_FILE=${NXT_BUILD_DIR}/src/${NXT_NODE}/package.json NXT_NODE_EXPORTS="export UNIT_SRC_PATH=${PWD}/src \ && export UNIT_BUILD_PATH=${PWD}/${NXT_BUILD_DIR} \ - && export UNIT_LIB_STATIC_PATH=${PWD}/${NXT_BUILD_DIR}/libunit.a" + && export UNIT_LIB_STATIC_PATH=${PWD}/${NXT_BUILD_DIR}/lib/libunit.a" if [ -n "$NXT_NODE_LOCAL" ]; then NXT_NODE_INSTALL=local-install @@ -149,7 +149,7 @@ cat << END >> $NXT_MAKEFILE .PHONY: ${NXT_NODE}-build .PHONY: ${NXT_NODE}-publish -${NXT_NODE}: ${NXT_NODE}-copy $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC +${NXT_NODE}: ${NXT_NODE}-copy $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC ${NXT_NODE_EXPORTS} && \\ cd ${NXT_NODE_TMP} && ${NXT_NODE_GYP} configure build clean @@ -201,7 +201,7 @@ ${NXT_NODE}-local-check: exit 1) ${NXT_NODE}-local-install: ${NXT_NODE_TARBALL} ${NXT_NODE}-local-check \ - $NXT_BUILD_DIR/$NXT_LIB_UNIT_STATIC + $NXT_BUILD_DIR/lib/$NXT_LIB_UNIT_STATIC ${NXT_NODE_EXPORTS} && \\ mkdir -p \$(DESTDIR)${NXT_NODE_LOCAL} && \\ cd \$(DESTDIR)${NXT_NODE_LOCAL} && \\ diff --git a/auto/modules/perl b/auto/modules/perl index 2daebd0d..3c88ef0e 100644 --- a/auto/modules/perl +++ b/auto/modules/perl @@ -185,25 +185,24 @@ cat << END >> $NXT_MAKEFILE all: ${NXT_PERL_MODULE} -${NXT_PERL_MODULE}: $NXT_BUILD_DIR/${NXT_PERL_MODULE}.unit.so +${NXT_PERL_MODULE}: $NXT_BUILD_DIR/lib/unit/modules/${NXT_PERL_MODULE}.unit.so -$NXT_BUILD_DIR/${NXT_PERL_MODULE}.unit.so: $nxt_objs - \$(NXT_MODULE_LINK) -o $NXT_BUILD_DIR/${NXT_PERL_MODULE}.unit.so \\ - $nxt_objs $NXT_PERL_LDOPTS $NXT_LD_OPT +$NXT_BUILD_DIR/lib/unit/modules/${NXT_PERL_MODULE}.unit.so: $nxt_objs + \$(NXT_MODULE_LINK) -o \$@ $nxt_objs $NXT_PERL_LDOPTS $NXT_LD_OPT install: ${NXT_PERL_MODULE}-install ${NXT_PERL_MODULE}-install: ${NXT_PERL_MODULE} install-check - install -d \$(DESTDIR)$NXT_MODULES - install -p $NXT_BUILD_DIR/${NXT_PERL_MODULE}.unit.so \\ - \$(DESTDIR)$NXT_MODULES/ + install -d \$(DESTDIR)$NXT_MODULESDIR + install -p $NXT_BUILD_DIR/lib/unit/modules/${NXT_PERL_MODULE}.unit.so \\ + \$(DESTDIR)$NXT_MODULESDIR/ uninstall: ${NXT_PERL_MODULE}-uninstall ${NXT_PERL_MODULE}-uninstall: - rm -f \$(DESTDIR)$NXT_MODULES/${NXT_PERL_MODULE}.unit.so - @rmdir -p \$(DESTDIR)$NXT_MODULES 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_MODULESDIR/${NXT_PERL_MODULE}.unit.so + @rmdir -p \$(DESTDIR)$NXT_MODULESDIR 2>/dev/null || true END diff --git a/auto/modules/php b/auto/modules/php index f0ecb709..a0f5379c 100644 --- a/auto/modules/php +++ b/auto/modules/php @@ -265,25 +265,25 @@ cat << END >> $NXT_MAKEFILE all: ${NXT_PHP_MODULE} -${NXT_PHP_MODULE}: $NXT_BUILD_DIR/${NXT_PHP_MODULE}.unit.so +${NXT_PHP_MODULE}: $NXT_BUILD_DIR/lib/unit/modules/${NXT_PHP_MODULE}.unit.so -$NXT_BUILD_DIR/${NXT_PHP_MODULE}.unit.so: $nxt_objs - \$(NXT_MODULE_LINK) -o $NXT_BUILD_DIR/${NXT_PHP_MODULE}.unit.so \\ +$NXT_BUILD_DIR/lib/unit/modules/${NXT_PHP_MODULE}.unit.so: $nxt_objs + \$(NXT_MODULE_LINK) -o \$@ \\ $nxt_objs ${NXT_PHP_LIB} ${NXT_PHP_LDFLAGS} $NXT_LD_OPT install: ${NXT_PHP_MODULE}-install ${NXT_PHP_MODULE}-install: ${NXT_PHP_MODULE} install-check - install -d \$(DESTDIR)$NXT_MODULES - install -p $NXT_BUILD_DIR/${NXT_PHP_MODULE}.unit.so \\ - \$(DESTDIR)$NXT_MODULES/ + install -d \$(DESTDIR)$NXT_MODULESDIR + install -p $NXT_BUILD_DIR/lib/unit/modules/${NXT_PHP_MODULE}.unit.so \\ + \$(DESTDIR)$NXT_MODULESDIR/ uninstall: ${NXT_PHP_MODULE}-uninstall ${NXT_PHP_MODULE}-uninstall: - rm -f \$(DESTDIR)$NXT_MODULES/${NXT_PHP_MODULE}.unit.so - @rmdir -p \$(DESTDIR)$NXT_MODULES 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_MODULESDIR/${NXT_PHP_MODULE}.unit.so + @rmdir -p \$(DESTDIR)$NXT_MODULESDIR 2>/dev/null || true END diff --git a/auto/modules/python b/auto/modules/python index 480ae1da..dfd632a1 100644 --- a/auto/modules/python +++ b/auto/modules/python @@ -157,7 +157,7 @@ for p in sys.path: print("};\n\n") -' > $NXT_BUILD_DIR/$NXT_PYTHON_MOUNTS_HEADER +' > $NXT_BUILD_DIR/include/$NXT_PYTHON_MOUNTS_HEADER $echo " + Python module: ${NXT_PYTHON_MODULE}.unit.so" @@ -213,25 +213,25 @@ cat << END >> $NXT_MAKEFILE all: ${NXT_PYTHON_MODULE} -${NXT_PYTHON_MODULE}: $NXT_BUILD_DIR/${NXT_PYTHON_MODULE}.unit.so +${NXT_PYTHON_MODULE}: $NXT_BUILD_DIR/lib/unit/modules/${NXT_PYTHON_MODULE}.unit.so -$NXT_BUILD_DIR/${NXT_PYTHON_MODULE}.unit.so: $nxt_objs - \$(NXT_MODULE_LINK) -o $NXT_BUILD_DIR/${NXT_PYTHON_MODULE}.unit.so \\ +$NXT_BUILD_DIR/lib/unit/modules/${NXT_PYTHON_MODULE}.unit.so: $nxt_objs + \$(NXT_MODULE_LINK) -o \$@ \\ $nxt_objs $NXT_PYTHON_LIBS $NXT_PYTHON_LDFLAGS $NXT_LD_OPT install: ${NXT_PYTHON_MODULE}-install ${NXT_PYTHON_MODULE}-install: ${NXT_PYTHON_MODULE} install-check - install -d \$(DESTDIR)$NXT_MODULES - install -p $NXT_BUILD_DIR/${NXT_PYTHON_MODULE}.unit.so \\ - \$(DESTDIR)$NXT_MODULES/ + install -d \$(DESTDIR)$NXT_MODULESDIR + install -p $NXT_BUILD_DIR/lib/unit/modules/${NXT_PYTHON_MODULE}.unit.so \\ + \$(DESTDIR)$NXT_MODULESDIR/ uninstall: ${NXT_PYTHON_MODULE}-uninstall ${NXT_PYTHON_MODULE}-uninstall: - rm -f \$(DESTDIR)$NXT_MODULES/${NXT_PYTHON_MODULE}.unit.so - @rmdir -p \$(DESTDIR)$NXT_MODULES 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_MODULESDIR/${NXT_PYTHON_MODULE}.unit.so + @rmdir -p \$(DESTDIR)$NXT_MODULESDIR 2>/dev/null || true END diff --git a/auto/modules/ruby b/auto/modules/ruby index 608193a6..7a7c9bd3 100644 --- a/auto/modules/ruby +++ b/auto/modules/ruby @@ -176,7 +176,7 @@ fi NXT_RUBY_MOUNTS_HEADER=nxt_${NXT_RUBY_MODULE}_mounts.h -NXT_RUBY_MOUNTS_PATH=$NXT_BUILD_DIR/$NXT_RUBY_MOUNTS_HEADER +NXT_RUBY_MOUNTS_PATH=$NXT_BUILD_DIR/include/$NXT_RUBY_MOUNTS_HEADER cat << END > $NXT_RUBY_MOUNTS_PATH @@ -234,7 +234,7 @@ $NXT_BUILD_DIR/$nxt_obj: $nxt_src $NXT_VERSION_H \$(CC) -c \$(CFLAGS) $NXT_RUBY_CFLAGS -DNXT_RUBY_MOUNTS_H=\"$NXT_RUBY_MOUNTS_HEADER\" \\ \$(NXT_INCS) $NXT_RUBY_INCPATH \\ $nxt_dep_flags \\ - -o $NXT_BUILD_DIR/$nxt_obj $nxt_src + -o \$@ $nxt_src $nxt_dep_post -include $NXT_BUILD_DIR/$nxt_dep @@ -251,25 +251,24 @@ cat << END >> $NXT_MAKEFILE all: ${NXT_RUBY_MODULE} -${NXT_RUBY_MODULE}: $NXT_BUILD_DIR/${NXT_RUBY_MODULE}.unit.so +${NXT_RUBY_MODULE}: $NXT_BUILD_DIR/lib/unit/modules/${NXT_RUBY_MODULE}.unit.so -$NXT_BUILD_DIR/${NXT_RUBY_MODULE}.unit.so: $nxt_objs - \$(NXT_MODULE_LINK) -o $NXT_BUILD_DIR/${NXT_RUBY_MODULE}.unit.so \\ - $nxt_objs $NXT_RUBY_LIBS $NXT_LD_OPT +$NXT_BUILD_DIR/lib/unit/modules/${NXT_RUBY_MODULE}.unit.so: $nxt_objs + \$(NXT_MODULE_LINK) -o \$@ $nxt_objs $NXT_RUBY_LIBS $NXT_LD_OPT install: ${NXT_RUBY_MODULE}-install ${NXT_RUBY_MODULE}-install: ${NXT_RUBY_MODULE} install-check - install -d \$(DESTDIR)$NXT_MODULES - install -p $NXT_BUILD_DIR/${NXT_RUBY_MODULE}.unit.so \\ - \$(DESTDIR)$NXT_MODULES/ + install -d \$(DESTDIR)$NXT_MODULESDIR + install -p $NXT_BUILD_DIR/lib/unit/modules/${NXT_RUBY_MODULE}.unit.so \\ + \$(DESTDIR)$NXT_MODULESDIR/ uninstall: ${NXT_RUBY_MODULE}-uninstall ${NXT_RUBY_MODULE}-uninstall: - rm -f \$(DESTDIR)$NXT_MODULES/${NXT_RUBY_MODULE}.unit.so - @rmdir -p \$(DESTDIR)$NXT_MODULES 2>/dev/null || true + rm -f \$(DESTDIR)$NXT_MODULESDIR/${NXT_RUBY_MODULE}.unit.so + @rmdir -p \$(DESTDIR)$NXT_MODULESDIR 2>/dev/null || true END diff --git a/auto/options b/auto/options index abcf531d..5487be7f 100644 --- a/auto/options +++ b/auto/options @@ -9,8 +9,6 @@ NXT_CFLAGS= NXT_CC_OPT= NXT_LD_OPT= -NXT_PREFIX= - NXT_DEBUG=NO NXT_INET6=YES @@ -44,6 +42,8 @@ NXT_TEST_BUILD_HPUX_SENDFILE=NO NXT_TESTS=NO +NXT_HELP=NO + for nxt_option do case "$nxt_option" in @@ -57,14 +57,35 @@ do --ld-opt=*) NXT_LD_OPT="$value" ;; --prefix=*) NXT_PREFIX="$value" ;; + --exec-prefix=*) NXT_EXEC_PREFIX="$value" ;; --bindir=*) NXT_BINDIR="$value" ;; --sbindir=*) NXT_SBINDIR="$value" ;; + --includedir=*) NXT_INCLUDEDIR="$value" ;; + --incdir=*) + >&2 echo "[warn] option --incdir is deprecated; use --includedir" + NXT_INCLUDEDIR="$value" + ;; --libdir=*) NXT_LIBDIR="$value" ;; - --incdir=*) NXT_INCDIR="$value" ;; + --modulesdir=*) NXT_MODULESDIR="$value" ;; + --modules=*) + >&2 echo "[warn] option --modules is deprecated; use --modulesdir" + NXT_MODULESDIR="$value" + ;; + --datarootdir=*) NXT_DATAROOTDIR="$value" ;; --mandir=*) NXT_MANDIR="$value" ;; - --modules=*) NXT_MODULES="$value" ;; - --state=*) NXT_STATE="$value" ;; - --tmp=*) NXT_TMP="$value" ;; + --localstatedir=*) NXT_LOCALSTATEDIR="$value" ;; + --statedir=*) NXT_STATEDIR="$value" ;; + --state=*) + >&2 echo "[warn] option --state is deprecated; use --statedir" + NXT_STATEDIR="$value" + ;; + --logdir=*) NXT_LOGDIR="$value" ;; + --runstatedir=*) NXT_RUNSTATEDIR="$value" ;; + --tmpdir=*) NXT_TMPDIR="$value" ;; + --tmp=*) + >&2 echo "[warn] option --tmp is deprecated; use --tmpdir" + NXT_TMPDIR="$value" + ;; --pid=*) NXT_PID="$value" ;; --log=*) NXT_LOG="$value" ;; @@ -124,63 +145,26 @@ do done -case "$NXT_PREFIX" in - ""|*/) ;; - *) NXT_PREFIX="$NXT_PREFIX/" ;; -esac - -case "$NXT_BINDIR" in - /*) ;; - *) NXT_BINDIR="$NXT_PREFIX$NXT_BINDIR" ;; -esac - -case "$NXT_SBINDIR" in - /*) ;; - *) NXT_SBINDIR="$NXT_PREFIX$NXT_SBINDIR" ;; -esac - -case "$NXT_LIBDIR" in - /*) ;; - *) NXT_LIBDIR="$NXT_PREFIX$NXT_LIBDIR" ;; -esac - -case "$NXT_INCDIR" in - /*) ;; - *) NXT_INCDIR="$NXT_PREFIX$NXT_INCDIR" ;; -esac - -case "$NXT_MANDIR" in - /*) ;; - *) NXT_MANDIR="$NXT_PREFIX$NXT_MANDIR" ;; -esac - -case "$NXT_MODULES" in - /*) ;; - *) NXT_MODULES="$NXT_PREFIX$NXT_MODULES" ;; -esac - -case "$NXT_STATE" in - /*) ;; - *) NXT_STATE="$NXT_PREFIX$NXT_STATE" ;; -esac - -case "$NXT_TMP" in - /*) ;; - *) NXT_TMP="$NXT_PREFIX$NXT_TMP" ;; -esac - -case "$NXT_PID" in - /*) ;; - *) NXT_PID="$NXT_PREFIX$NXT_PID" ;; -esac - -case "$NXT_LOG" in - /*) ;; - *) NXT_LOG="$NXT_PREFIX$NXT_LOG" ;; -esac - -case "$NXT_CONTROL" in - unix:/*) ;; - unix:*) NXT_CONTROL="unix:$NXT_PREFIX${NXT_CONTROL##unix:}" ;; - *) ;; -esac +NXT_PREFIX="${NXT_PREFIX-"/usr/local"}" + +NXT_EXEC_PREFIX="${NXT_EXEC_PREFIX-"$NXT_PREFIX"}" +NXT_BINDIR="${NXT_BINDIR-"$NXT_EXEC_PREFIX/bin"}" +NXT_SBINDIR="${NXT_SBINDIR-"$NXT_EXEC_PREFIX/sbin"}" + +NXT_INCLUDEDIR="${NXT_INCLUDEDIR-"$NXT_PREFIX/include"}" + +NXT_LIBDIR="${NXT_LIBDIR-"$NXT_PREFIX/lib"}" +NXT_MODULESDIR="${NXT_MODULESDIR-"$NXT_LIBDIR/unit/modules"}" + +NXT_DATAROOTDIR="${NXT_DATAROOTDIR-"$NXT_PREFIX/share"}" +NXT_MANDIR="${NXT_MANDIR-"$NXT_DATAROOTDIR/man"}" + +NXT_LOCALSTATEDIR="${NXT_LOCALSTATEDIR-"$NXT_PREFIX/var"}" +NXT_STATEDIR="${NXT_STATEDIR-"$NXT_LOCALSTATEDIR/lib/unit"}" +NXT_LOGDIR="${NXT_LOGDIR-"$NXT_LOCALSTATEDIR/log/unit"}" +NXT_LOG="${NXT_LOG-"$NXT_LOGDIR/unit.log"}" +NXT_RUNSTATEDIR="${NXT_RUNSTATEDIR-"$NXT_LOCALSTATEDIR/run/unit"}" +NXT_CONTROL="${NXT_CONTROL-"unix:$NXT_RUNSTATEDIR/control.unit.sock"}" +NXT_PID="${NXT_PID-"$NXT_RUNSTATEDIR/unit.pid"}" + +NXT_TMPDIR="${NXT_TMPDIR-"/tmp"}" diff --git a/auto/os/conf b/auto/os/conf index 02c4afaf..bc1f5ef7 100644 --- a/auto/os/conf +++ b/auto/os/conf @@ -33,7 +33,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -57,7 +57,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -88,7 +88,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -117,7 +117,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.dylib" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.dylib" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.dylib" NXT_LIB_UNIT_STATIC="libunit.a" @@ -141,7 +141,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -163,7 +163,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -185,7 +185,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -206,7 +206,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -227,7 +227,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -248,7 +248,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -268,7 +268,7 @@ case "$NXT_SYSTEM" in NXT_LIB_STATIC="libnxt.a" NXT_LIB_SHARED="libnxt.so" - NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/libnxt.so" + NXT_LIB_SHARED_LOCAL="$NXT_BUILD_DIR/lib/libnxt.so" NXT_LIB_UNIT_STATIC="libunit.a" @@ -21,14 +21,14 @@ NXT_TEST_LIBS='$NXT_TEST_LIBS' NXT_LIBRT='$NXT_LIBRT' -echo=$NXT_BUILD_DIR/echo +echo=$NXT_BUILD_DIR/bin/echo NXT_LIB_AUX_CFLAGS= NXT_LIB_AUX_LIBS= NXT_LIB_UNIT_STATIC='$NXT_LIB_UNIT_STATIC' -NXT_MODULES='$NXT_MODULES' -NXT_TMP='$NXT_TMP' +NXT_MODULESDIR='$NXT_MODULESDIR' +NXT_TMPDIR='$NXT_TMPDIR' END diff --git a/auto/sources b/auto/sources index 2ca78844..f4a7170a 100644 --- a/auto/sources +++ b/auto/sources @@ -92,6 +92,7 @@ NXT_LIB_SRCS=" \ src/nxt_http_error.c \ src/nxt_http_route.c \ src/nxt_http_route_addr.c \ + src/nxt_http_rewrite.c \ src/nxt_http_return.c \ src/nxt_http_static.c \ src/nxt_http_proxy.c \ @@ -108,19 +109,6 @@ NXT_LIB_SRCS=" \ src/nxt_fs.c \ " -NXT_LIB_SRC0=" \ - src/nxt_buf_filter.c \ - src/nxt_job_file.c \ - src/nxt_stream_module.c \ - src/nxt_stream_source.c \ - src/nxt_upstream_source.c \ - src/nxt_http_source.c \ - src/nxt_fastcgi_source.c \ - src/nxt_fastcgi_record_parse.c \ -\ - src/nxt_mem_pool_cleanup.h \ - src/nxt_mem_pool_cleanup.c \ -" NXT_LIB_UNIT_SRCS="src/nxt_unit.c" @@ -136,7 +124,7 @@ NXT_LIB_PCRE_SRCS="src/nxt_pcre.c" NXT_LIB_PCRE2_SRCS="src/nxt_pcre2.c" if [ "$NXT_NJS" != "NO" ]; then - NXT_LIB_SRCS="$NXT_LIB_SRCS src/nxt_js.c src/nxt_http_js.c" + NXT_LIB_SRCS="$NXT_LIB_SRCS src/nxt_js.c src/nxt_http_js.c src/nxt_script.c" fi NXT_LIB_EPOLL_SRCS="src/nxt_epoll_engine.c" diff --git a/auto/summary b/auto/summary index 51db0eae..fabe3b10 100644 --- a/auto/summary +++ b/auto/summary @@ -10,11 +10,11 @@ Unit configuration summary: bin directory: ............. "$NXT_BINDIR" sbin directory: ............ "$NXT_SBINDIR" lib directory: ............. "$NXT_LIBDIR" - include directory: ......... "$NXT_INCDIR" + include directory: ......... "$NXT_INCLUDEDIR" man pages directory: ....... "$NXT_MANDIR" - modules directory: ......... "$NXT_MODULES" - state directory: ........... "$NXT_STATE" - tmp directory: ............. "$NXT_TMP" + modules directory: ......... "$NXT_MODULESDIR" + state directory: ........... "$NXT_STATEDIR" + tmp directory: ............. "$NXT_TMPDIR" pid file: .................. "$NXT_PID" log file: .................. "$NXT_LOG" @@ -24,24 +24,13 @@ NXT_BUILD_DIR=${NXT_BUILD_DIR:-build} NXT_AUTOTEST=$NXT_BUILD_DIR/autotest NXT_AUTOCONF_ERR=$NXT_BUILD_DIR/autoconf.err NXT_AUTOCONF_DATA=$NXT_BUILD_DIR/autoconf.data -NXT_AUTO_CONFIG_H=$NXT_BUILD_DIR/nxt_auto_config.h -NXT_VERSION_H=$NXT_BUILD_DIR/nxt_version.h +NXT_AUTO_CONFIG_H=$NXT_BUILD_DIR/include/nxt_auto_config.h +NXT_VERSION_H=$NXT_BUILD_DIR/include/nxt_version.h NXT_MAKEFILE=$NXT_BUILD_DIR/Makefile CC=${CC:-cc} NXT_DAEMON=unitd -NXT_BINDIR="bin" -NXT_SBINDIR="sbin" -NXT_LIBDIR="lib" -NXT_INCDIR="include" -NXT_MANDIR="share/man" -NXT_MODULES="modules" -NXT_STATE="state" -NXT_TMP="tmp" -NXT_PID="unit.pid" -NXT_LOG="unit.log" -NXT_CONTROL="unix:control.unit.sock" NXT_USER="nobody" NXT_GROUP= @@ -66,7 +55,19 @@ esac . auto/os/test . auto/options -test -d $NXT_BUILD_DIR || mkdir $NXT_BUILD_DIR +mkdir -p $NXT_BUILD_DIR +mkdir -p $NXT_BUILD_DIR/bin +mkdir -p $NXT_BUILD_DIR/include +mkdir -p $NXT_BUILD_DIR/lib +mkdir -p $NXT_BUILD_DIR/lib/unit/modules +mkdir -p $NXT_BUILD_DIR/sbin +mkdir -p $NXT_BUILD_DIR/share/man/man8 +mkdir -p $NXT_BUILD_DIR/src +mkdir -p $NXT_BUILD_DIR/src/test +mkdir -p $NXT_BUILD_DIR/var/lib/unit +mkdir -p $NXT_BUILD_DIR/var/log/unit +mkdir -p $NXT_BUILD_DIR/var/run/unit + > $NXT_AUTOCONF_ERR > $NXT_AUTO_CONFIG_H @@ -82,9 +83,9 @@ cat << END >> $NXT_AUTO_CONFIG_H #define NXT_PID "$NXT_PID" #define NXT_LOG "$NXT_LOG" -#define NXT_MODULES "$NXT_MODULES" -#define NXT_STATE "$NXT_STATE" -#define NXT_TMP "$NXT_TMP" +#define NXT_MODULESDIR "$NXT_MODULESDIR" +#define NXT_STATEDIR "$NXT_STATEDIR" +#define NXT_TMPDIR "$NXT_TMPDIR" #define NXT_CONTROL_SOCK "$NXT_CONTROL" diff --git a/docs/changes.xml b/docs/changes.xml index 91ad1966..68257aa8 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -14,6 +14,117 @@ unit-ruby unit-jsc-common unit-jsc8 unit-jsc10 unit-jsc11 unit-jsc13 unit-jsc14 unit-jsc15 unit-jsc16 unit-jsc17 unit-jsc18 + unit-jsc19 unit-jsc20" + ver="1.30.0" rev="1" + date="2023-05-10" time="18:00:00 +0300" + packager="Nginx Packaging <nginx-packaging@f5.com>"> + +<change> +<para> +NGINX Unit updated to 1.30.0. +</para> +</change> + +</changes> + + +<changes apply="unit-jsc20" ver="1.30.0" rev="1" + date="2023-05-03" time="15:00:00 -0700" + packager="Nginx Packaging <nginx-packaging@f5.com>"> + +<change> +<para> +Initial release of Java 20 module for NGINX Unit. +</para> +</change> + +</changes> + + +<changes apply="unit" ver="1.30.0" rev="1" + date="2023-05-10" time="18:00:00 +0300" + packager="Nginx Packaging <nginx-packaging@f5.com>"> + +<change type="change"> +<para> +remove Unix domain listen sockets upon reconfiguration. +</para> +</change> + +<change type="feature"> +<para> +basic URI rewrite support. +</para> +</change> + +<change type="feature"> +<para> +NJS loadable modules support. +</para> +</change> + +<change type="feature"> +<para> +per-application logging. +</para> +</change> + +<change type="feature"> +<para> +conditional logging of route selection. +</para> +</change> + +<change type="feature"> +<para> +support the keys API on the request objects in NJS. +</para> +</change> + +<change type="feature"> +<para> +default values for 'make install' pathnames such as prefix; +this allows to './configure && make && sudo make install'. +</para> +</change> + +<change type="feature"> +<para> +"server_version" setting to omit the version token from "Server" header field. +</para> +</change> + +<change type="bugfix"> +<para> +request header field values could be corrupted in some cases; the bug had +appeared in 1.29.0. +</para> +</change> + +<change type="bugfix"> +<para> +PHP error handling (added missing 403 and 404 errors). +</para> +</change> + +<change type="bugfix"> +<para> +Perl applications crash on second responder call. +</para> +</change> + +</changes> + + +<changes apply="unit-php + unit-python unit-python2.7 + unit-python3.4 unit-python3.5 unit-python3.6 unit-python3.7 + unit-python3.8 unit-python3.9 unit-python3.10 unit-python3.11 + unit-go + unit-perl + unit-ruby + unit-jsc-common unit-jsc8 unit-jsc10 unit-jsc11 unit-jsc13 + unit-jsc14 unit-jsc15 unit-jsc16 unit-jsc17 unit-jsc18 unit-jsc19" ver="1.29.1" rev="1" date="2023-02-28" time="18:00:00 +0300" diff --git a/docs/man/unitd.8.in b/docs/man/man8/unitd.8.in index 46182781..a43e671f 100644 --- a/docs/man/unitd.8.in +++ b/docs/man/man8/unitd.8.in @@ -1,36 +1,36 @@ .\" (C) 2017-2021, NGINX, Inc. .\" -.Dd March 16, 2021 -.Dt UNITD 8 -.Os -.Sh NAME +.Dd 2023-04-26 +.Dt unitd 8 +.Os NGINX Unit +.Sh Name .Nm unitd .Nd "runs the NGINX Unit daemon" -.Sh SYNOPSIS +.Sh Synopsis .Nm .Op Fl Fl no-daemon .Op Fl Fl control Ar socket .Op Fl Fl group Ar name .Op Fl Fl user Ar name .Op Fl Fl log Ar file -.Op Fl Fl modules Ar directory +.Op Fl Fl modulesdir Ar directory .Op Fl Fl pid Ar file -.Op Fl Fl state Ar directory +.Op Fl Fl statedir Ar directory .Nm -.Op Fl h | Fl Fl help | v | Fl Fl version -.Sh DESCRIPTION +.Op Fl h | Fl Fl help | Fl Fl version +.Sh Description NGINX Unit is a polyglot app server, a reverse proxy, and a static file server for UNIX-like systems. It was built by .Xr nginx 8 team members from scratch to be highly efficient and fully configurable at runtime. -.Sh OPTIONS +.Sh Options .Bl -tag -width indent .It Fl h , Fl Fl help Displays a summary of Unit's command-line options and their compile-time defaults. -.It Fl v , Fl Fl version +.It Fl Fl version Displays Unit's version and the .Pa ./configure settings it was built with. @@ -43,38 +43,36 @@ or UNIX-domain format. Override group name and user name used to run Unit's non-privileged processes. .It Fl Fl log Ar file Overrides the pathname for Unit's log. -.It Fl Fl modules Ar directory +.It Fl Fl modulesdir Ar directory Overrides the directory path for Unit's language modules .Po Pa *.unit.so .Pc files . .It Fl Fl pid Ar file Overrides the pathname for the PID file of Unit's main process. -.It Fl Fl state Ar directory +.It Fl Fl statedir Ar directory Overrides the directory path for Unit's state storage. .El -.Sh EXIT STATUS +.Sh Exit status Exit status is 0 on success, or 1 if the daemon encounters an error. -.Sh FILES +.Sh Files .Bl -tag -width indent .It Pa %%PID_PATH%% The PID file of Unit's main process. .It Pa %%ERROR_LOG_PATH%% A general-purpose log for diagnostics and troubleshooting. .El -.Sh SOCKETS +.Sh Sockets .Bl -tag -width indent .It Pa %%SOCKET_PATH%% The socket address of Unit's control API. .El -.Sh AUTHORS -(C) 2017-2021, NGINX, Inc. -.Sh SEE ALSO +.Sh Copyright +(C) 2017-2023, NGINX, Inc. .Pp -Website: -.Pa https://unit.nginx.org +SPDX-License-Identifier: Apache-2.0 +.Sh See also +.Lk https://unit.nginx.org Website .Pp -User mailing list: -.Pa https://mailman.nginx.org/mailman/listinfo/unit +.Lk https://mailman.nginx.org/mailman/listinfo/unit "User mailing list" .Pp -GitHub: -.Pa https://github.com/nginx/unit +.Lk https://github.com/nginx/unit GitHub diff --git a/pkg/contrib/src/njs/Makefile b/pkg/contrib/src/njs/Makefile index 07740895..6a4fdf9d 100644 --- a/pkg/contrib/src/njs/Makefile +++ b/pkg/contrib/src/njs/Makefile @@ -15,5 +15,8 @@ njs: njs-$(NJS_VERSION).tar.gz .sum-njs $(MOVE) .njs: njs - cd $< && ./configure --no-libxml2 && $(MAKE) libnjs + cd $< && ./configure \ + --no-libxml2 \ + --no-zlib \ + && $(MAKE) libnjs touch $@ diff --git a/pkg/contrib/src/njs/SHA512SUMS b/pkg/contrib/src/njs/SHA512SUMS index 0bba673b..c94e5638 100644 --- a/pkg/contrib/src/njs/SHA512SUMS +++ b/pkg/contrib/src/njs/SHA512SUMS @@ -1 +1 @@ -5063fcfac18298d86157d05dc618f47815763a2192538befa5f046d081a7d5c6b624b65258674a6d9719147c102a703d5c3a80d937f4e9d43985da8e85dbc539 njs-0.7.10.tar.gz +9cac2ced65bbfd712f7797f2bfa3fb20509a7e7bd68e8621d5fad32270f6d20a015d707665222559a72f525618bc91e09986a7bedce28af5f0fec9c20be41452 njs-0.7.12.tar.gz diff --git a/pkg/contrib/src/njs/version b/pkg/contrib/src/njs/version index 6088389d..64999f82 100644 --- a/pkg/contrib/src/njs/version +++ b/pkg/contrib/src/njs/version @@ -1 +1 @@ -NJS_VERSION := 0.7.10 +NJS_VERSION := 0.7.12 diff --git a/pkg/deb/Makefile b/pkg/deb/Makefile index 580cb655..09544892 100644 --- a/pkg/deb/Makefile +++ b/pkg/deb/Makefile @@ -19,6 +19,21 @@ BUILD_DEPENDS = $(BUILD_DEPENDS_unit) MODULES= +# Ubuntu 23.04 +ifeq ($(CODENAME),lunar) +include Makefile.php +include Makefile.python311 +include Makefile.go +include Makefile.perl +include Makefile.ruby +include Makefile.jsc-common +include Makefile.jsc11 +include Makefile.jsc17 +include Makefile.jsc18 +include Makefile.jsc19 +include Makefile.jsc20 +endif + # Ubuntu 22.10 ifeq ($(CODENAME),kinetic) include Makefile.php @@ -117,11 +132,11 @@ endif CONFIGURE_ARGS_COMMON=\ --prefix=/usr \ - --state=/var/lib/unit \ + --statedir=/var/lib/unit \ --control="unix:/var/run/control.unit.sock" \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --tests \ @@ -199,7 +214,8 @@ endif debuild/unit_$(VERSION).orig.tar.gz: | debuild/$(SRCDIR)/debian cd ../.. && tar -czf pkg/deb/debuild/$(SRCDIR).tar.gz \ --transform "s#^#$(SRCDIR)/#" \ - LICENSE NOTICE CHANGES README.md CONTRIBUTING.md configure auto src test version go pkg/contrib docs/man/unitd.8.in + LICENSE NOTICE CHANGES README.md CONTRIBUTING.md configure auto src \ + test tools version go pkg/contrib docs/man/man8/unitd.8.in mv debuild/$(SRCDIR).tar.gz debuild/unit_$(VERSION).orig.tar.gz cd debuild && tar zxf unit_$(VERSION).orig.tar.gz @@ -288,8 +304,8 @@ test: unit modules for so in `find debuild-*/unit-$(VERSION)/debian/build-unit/ -type f \( -name "*.so" -o -name "*.jar" \)`; do \ soname=`basename $${so}` ; \ test "$${soname}" = "java.unit.so" && continue ; \ - test -h debuild/unit-$(VERSION)/debian/build-unit/build/$${soname} || \ - ln -fs `pwd`/$${so} debuild/unit-$(VERSION)/debian/build-unit/build/$${soname} ; \ + test -h debuild/unit-$(VERSION)/debian/build-unit/build/lib/$${soname} || \ + ln -fs `pwd`/$${so} debuild/unit-$(VERSION)/debian/build-unit/build/lib/$${soname} ; \ done ; \ ( cd debuild/unit-$(VERSION)/debian/build-unit && env python3 -m pytest --user=nobody $(PYTEST_ARGS) ) ; \ } @@ -299,8 +315,8 @@ test-debug: unit modules for so in `find debuild-*/unit-$(VERSION)/debian/build-unit-debug/ -type f \( -name "*.so" -o -name "*.jar" \)`; do \ soname=`basename $${so}` ; \ test "$${soname}" = "java.unit.so" && continue ; \ - test -h debuild/unit-$(VERSION)/debian/build-unit-debug/build/$${soname} || \ - ln -fs `pwd`/$${so} debuild/unit-$(VERSION)/debian/build-unit-debug/build/$${soname} ; \ + test -h debuild/unit-$(VERSION)/debian/build-unit-debug/build/lib/$${soname} || \ + ln -fs `pwd`/$${so} debuild/unit-$(VERSION)/debian/build-unit-debug/build/lib/$${soname} ; \ done ; \ ( cd debuild/unit-$(VERSION)/debian/build-unit-debug && env python3 -m pytest --user=nobody $(PYTEST_ARGS) ) ; \ } diff --git a/pkg/deb/Makefile.jsc20 b/pkg/deb/Makefile.jsc20 new file mode 100644 index 00000000..5b7c6d9f --- /dev/null +++ b/pkg/deb/Makefile.jsc20 @@ -0,0 +1,71 @@ +MODULES+= jsc20 +MODULE_SUFFIX_jsc20= jsc20 + +MODULE_SUMMARY_jsc20= Java 20 module for NGINX Unit + +MODULE_VERSION_jsc20= $(VERSION) +MODULE_RELEASE_jsc20= 1 + +MODULE_CONFARGS_jsc20= java --module=java20 --home=/usr/lib/jvm/java-20-openjdk-$$\(DEB_HOST_ARCH\) --jars=/usr/share/unit-jsc-common/ +MODULE_MAKEARGS_jsc20= java20 +MODULE_INSTARGS_jsc20= java20-install + +MODULE_SOURCES_jsc20= unit.example-jsc-app \ + unit.example-jsc20-config + +BUILD_DEPENDS_jsc20= openjdk-20-jdk-headless openjdk-20-jre-headless +BUILD_DEPENDS+= $(BUILD_DEPENDS_jsc20) + +MODULE_BUILD_DEPENDS_jsc20=,openjdk-20-jdk-headless +MODULE_DEPENDS_jsc20=,openjdk-20-jre-headless,unit-jsc-common (= $(MODULE_VERSION_jsc_common)-$(MODULE_RELEASE_jsc_common)~$(CODENAME)) + +define MODULE_PREINSTALL_jsc20 + mkdir -p debian/unit-jsc20/usr/share/doc/unit-jsc20/examples/jsc-app + install -m 644 -p debian/unit.example-jsc-app debian/unit-jsc20/usr/share/doc/unit-jsc20/examples/jsc-app/index.jsp + install -m 644 -p debian/unit.example-jsc20-config debian/unit-jsc20/usr/share/doc/unit-jsc20/examples/unit.config + install -m 644 -p src/java/README.JSR-340 debian/unit-jsc20/usr/share/doc/unit-jsc20/ +endef +export MODULE_PREINSTALL_jsc20 + +define MODULE_POSTINSTALL_jsc20 + cd $$\(BUILDDIR_unit\) \&\& \ + DESTDIR=$$\(INSTALLDIR\) make java-shared-uninstall +endef +export MODULE_POSTINSTALL_jsc20 + +define MODULE_POST_jsc20 +cat <<BANNER +---------------------------------------------------------------------- + +The $(MODULE_SUMMARY_jsc20) has been installed. + +To check out the sample app, run these commands: + + sudo service unit restart + cd /usr/share/doc/unit-$(MODULE_SUFFIX_jsc20)/examples + sudo curl -X PUT --data-binary @unit.config --unix-socket /var/run/control.unit.sock http://localhost/config + curl http://localhost:8800/ + +Online documentation is available at https://unit.nginx.org + +NOTICE: + +This version of Unit code is made available in support of the open source +development process. This is an intermediate build made available for +testing purposes only. This Unit code is untested and presumed incompatible +with the JSR 340 Java Servlet 3.1 specification. You should not deploy or +write to this code. You should instead deploy and write production +applications on pre-built binaries that have been tested and certified +to meet the JSR-340 compatibility requirements such as certified binaries +published for the JSR-340 reference implementation available at +https://javaee.github.io/glassfish/. + +Redistribution of any Intermediate Build must retain this notice. + +Oracle and Java are registered trademarks of Oracle and/or its affiliates. +Other names may be trademarks of their respective owners. + +---------------------------------------------------------------------- +BANNER +endef +export MODULE_POST_jsc20 diff --git a/pkg/deb/Makefile.python311 b/pkg/deb/Makefile.python311 new file mode 100644 index 00000000..67fd3289 --- /dev/null +++ b/pkg/deb/Makefile.python311 @@ -0,0 +1,46 @@ +MODULES+= python311 +MODULE_SUFFIX_python311= python3.11 + +MODULE_SUMMARY_python311= Python 3.11 module for NGINX Unit + +MODULE_VERSION_python311= $(VERSION) +MODULE_RELEASE_python311= 1 + +MODULE_CONFARGS_python311= python --config=python3.11-config +MODULE_MAKEARGS_python311= python3.11 +MODULE_INSTARGS_python311= python3.11-install + +MODULE_SOURCES_python311= unit.example-python-app \ + unit.example-python3.11-config + +BUILD_DEPENDS_python311= python3.11-dev +BUILD_DEPENDS+= $(BUILD_DEPENDS_python311) + +MODULE_BUILD_DEPENDS_python311=,python3.11-dev + +define MODULE_PREINSTALL_python311 + mkdir -p debian/unit-python3.11/usr/share/doc/unit-python3.11/examples/python-app + install -m 644 -p debian/unit.example-python-app debian/unit-python3.11/usr/share/doc/unit-python3.11/examples/python-app/wsgi.py + install -m 644 -p debian/unit.example-python3.11-config debian/unit-python3.11/usr/share/doc/unit-python3.11/examples/unit.config +endef +export MODULE_PREINSTALL_python311 + +define MODULE_POST_python311 +cat <<BANNER +---------------------------------------------------------------------- + +The $(MODULE_SUMMARY_python311) has been installed. + +To check out the sample app, run these commands: + + sudo service unit restart + cd /usr/share/doc/unit-$(MODULE_SUFFIX_python311)/examples + sudo curl -X PUT --data-binary @unit.config --unix-socket /var/run/control.unit.sock http://localhost/config + curl http://localhost:8400/ + +Online documentation is available at https://unit.nginx.org + +---------------------------------------------------------------------- +BANNER +endef +export MODULE_POST_python311 diff --git a/pkg/deb/debian.module/rules-noarch.in b/pkg/deb/debian.module/rules-noarch.in index 0f05aaba..e56e06bc 100644 --- a/pkg/deb/debian.module/rules-noarch.in +++ b/pkg/deb/debian.module/rules-noarch.in @@ -35,7 +35,7 @@ configure.unit: config.env.unit cd $(BUILDDIR_unit) && \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/modules \ + --modulesdir=/usr/lib/unit/modules \ --cc-opt="$(CFLAGS)" && \ ./configure %%MODULE_CONFARGS%% touch $@ @@ -44,7 +44,7 @@ configure.unit_debug: config.env.unit_debug cd $(BUILDDIR_unit_debug) && \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/debug-modules \ + --modulesdir=/usr/lib/unit/debug-modules \ --cc-opt="$(CFLAGS)" \ --debug && \ ./configure %%MODULE_CONFARGS%% diff --git a/pkg/deb/debian.module/rules.in b/pkg/deb/debian.module/rules.in index f1217553..861a9c00 100755 --- a/pkg/deb/debian.module/rules.in +++ b/pkg/deb/debian.module/rules.in @@ -35,7 +35,7 @@ configure.unit: config.env.unit cd $(BUILDDIR_unit) && \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/modules \ + --modulesdir=/usr/lib/unit/modules \ --cc-opt="$(CFLAGS)" && \ ./configure %%MODULE_CONFARGS%% touch $@ @@ -44,7 +44,7 @@ configure.unit_debug: config.env.unit_debug cd $(BUILDDIR_unit_debug) && \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/debug-modules \ + --modulesdir=/usr/lib/unit/debug-modules \ --cc-opt="$(CFLAGS)" \ --debug && \ ./configure %%MODULE_CONFARGS%% diff --git a/pkg/deb/debian.module/unit.example-jsc20-config b/pkg/deb/debian.module/unit.example-jsc20-config new file mode 100644 index 00000000..57865a45 --- /dev/null +++ b/pkg/deb/debian.module/unit.example-jsc20-config @@ -0,0 +1,15 @@ +{ + "applications": { + "example_java20": { + "processes": 1, + "type": "java 20", + "webapp": "/usr/share/doc/unit-jsc20/examples/jsc-app" + } + }, + + "listeners": { + "*:8800": { + "pass": "applications/example_java20" + } + } +} diff --git a/pkg/deb/debian.module/unit.example-python3.11-config b/pkg/deb/debian.module/unit.example-python3.11-config new file mode 100644 index 00000000..39b31b57 --- /dev/null +++ b/pkg/deb/debian.module/unit.example-python3.11-config @@ -0,0 +1,16 @@ +{ + "applications": { + "example_python": { + "type": "python 3.11", + "processes": 2, + "path": "/usr/share/doc/unit-python3.11/examples/python-app", + "module": "wsgi" + } + }, + + "listeners": { + "*:8400": { + "pass": "applications/example_python" + } + } +} diff --git a/pkg/deb/debian/dirs b/pkg/deb/debian/dirs index 4a6618c8..2568bac1 100644 --- a/pkg/deb/debian/dirs +++ b/pkg/deb/debian/dirs @@ -1,3 +1,4 @@ +usr/bin usr/sbin usr/lib/unit var/lib/unit diff --git a/pkg/deb/debian/rules.in b/pkg/deb/debian/rules.in index 23812926..0d7cf830 100644 --- a/pkg/deb/debian/rules.in +++ b/pkg/deb/debian/rules.in @@ -32,6 +32,7 @@ config.env.%: njs cp -Pa $(CURDIR)/configure $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/src $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/test $(BUILDDIR_$*)/ + cp -Pa $(CURDIR)/tools $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/version $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/CHANGES $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/LICENSE $(BUILDDIR_$*)/ @@ -39,8 +40,8 @@ config.env.%: njs cp -Pa $(CURDIR)/README.md $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/CONTRIBUTING.md $(BUILDDIR_$*)/ cp -Pa $(CURDIR)/go $(BUILDDIR_$*)/ - mkdir -p $(BUILDDIR_$*)/docs/man - cp -Pa $(CURDIR)/docs/man/unitd.8.in $(BUILDDIR_$*)/docs/man/ + mkdir -p $(BUILDDIR_$*)/docs/man/man8 + cp -Pa $(CURDIR)/docs/man/man8/unitd.8.in $(BUILDDIR_$*)/docs/man/man8/ touch $@ configure.unit: config.env.unit @@ -48,7 +49,7 @@ configure.unit: config.env.unit PKG_CONFIG_PATH=$(CURDIR)/pkg/contrib/njs/build \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/modules \ + --modulesdir=/usr/lib/unit/modules \ --libdir=/usr/lib/$(DEB_HOST_MULTIARCH) \ --cc-opt="$(CFLAGS)" \ --ld-opt="$(LDFLAGS)" @@ -59,7 +60,7 @@ configure.unit_debug: config.env.unit_debug PKG_CONFIG_PATH=$(CURDIR)/pkg/contrib/njs/build \ CFLAGS= ./configure \ %%CONFIGURE_ARGS%% \ - --modules=/usr/lib/unit/debug-modules \ + --modulesdir=/usr/lib/unit/debug-modules \ --libdir=/usr/lib/$(DEB_HOST_MULTIARCH) \ --cc-opt="$(CFLAGS)" \ --ld-opt="$(LDFLAGS)" \ @@ -69,7 +70,7 @@ configure.unit_debug: config.env.unit_debug build-arch.%: configure.% dh_testdir $(MAKE) -C $(BUILDDIR_$*) - $(MAKE) -C $(BUILDDIR_$*) build/libunit.a + $(MAKE) -C $(BUILDDIR_$*) build/lib/libunit.a ifeq ($(DOTESTS), 1) $(MAKE) -C $(BUILDDIR_$*) tests endif @@ -112,8 +113,10 @@ install: build do.tests dh_installlogrotate cd $(BUILDDIR_unit) && DESTDIR=$(INSTALLDIR) make install cd $(BUILDDIR_unit) && DESTDIR=$(INSTALLDIR_dev) make libunit-install - install -m 755 $(BUILDDIR_unit_debug)/build/unitd $(INSTALLDIR)/usr/sbin/unitd-debug - install -m 644 $(BUILDDIR_unit_debug)/build/libunit.a $(INSTALLDIR_dev)/usr/lib/$(DEB_HOST_MULTIARCH)/libunit-debug.a + install -m 755 $(BUILDDIR_unit)/tools/unitc $(INSTALLDIR)/usr/bin/unitc + install -m 755 $(BUILDDIR_unit)/tools/setup-unit $(INSTALLDIR)/usr/bin/setup-unit + install -m 755 $(BUILDDIR_unit_debug)/build/sbin/unitd $(INSTALLDIR)/usr/sbin/unitd-debug + install -m 644 $(BUILDDIR_unit_debug)/build/lib/libunit.a $(INSTALLDIR_dev)/usr/lib/$(DEB_HOST_MULTIARCH)/libunit-debug.a mkdir -p $(INSTALLDIR)/usr/share/doc/unit/examples install -m 644 debian/unit.example.config $(INSTALLDIR)/usr/share/doc/unit/examples/example.config install -m 644 CHANGES $(INSTALLDIR)/usr/share/doc/unit/changelog diff --git a/pkg/docker/Dockerfile.go1.20 b/pkg/docker/Dockerfile.go1.20 index 408e0d2b..50b4d5b3 100644 --- a/pkg/docker/Dockerfile.go1.20 +++ b/pkg/docker/Dockerfile.go1.20 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure go --go-path=$GOPATH \ && make -j $NCPU go-install-src libunit-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure go --go-path=$GOPATH \ && make -j $NCPU go-install-src libunit-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.jsc11 b/pkg/docker/Dockerfile.jsc11 index dcadeef4..bd987ae5 100644 --- a/pkg/docker/Dockerfile.jsc11 +++ b/pkg/docker/Dockerfile.jsc11 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure java --jars=/usr/share/unit-jsc-common/ \ && make -j $NCPU java-shared-install java-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure java --jars=/usr/share/unit-jsc-common/ \ && make -j $NCPU java-shared-install java-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.minimal b/pkg/docker/Dockerfile.minimal index 2a2d2755..06a85b22 100644 --- a/pkg/docker/Dockerfile.minimal +++ b/pkg/docker/Dockerfile.minimal @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure \ && make -j $NCPU version \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure \ && make -j $NCPU version \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.node18 b/pkg/docker/Dockerfile.node18 index b782b60e..b3fb46d3 100644 --- a/pkg/docker/Dockerfile.node18 +++ b/pkg/docker/Dockerfile.node18 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure nodejs --node-gyp=/usr/local/lib/node_modules/npm/bin/node-gyp-bin/node-gyp \ && make -j $NCPU node node-install libunit-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure nodejs --node-gyp=/usr/local/lib/node_modules/npm/bin/node-gyp-bin/node-gyp \ && make -j $NCPU node node-install libunit-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.perl5.36 b/pkg/docker/Dockerfile.perl5.36 index c265bd36..2dc31e53 100644 --- a/pkg/docker/Dockerfile.perl5.36 +++ b/pkg/docker/Dockerfile.perl5.36 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure perl \ && make -j $NCPU perl-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure perl \ && make -j $NCPU perl-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.php8.2 b/pkg/docker/Dockerfile.php8.2 index 660cb2ba..fcf3f59e 100644 --- a/pkg/docker/Dockerfile.php8.2 +++ b/pkg/docker/Dockerfile.php8.2 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure php \ && make -j $NCPU php-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure php \ && make -j $NCPU php-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.python3.11 b/pkg/docker/Dockerfile.python3.11 index 10ec6518..89cd315a 100644 --- a/pkg/docker/Dockerfile.python3.11 +++ b/pkg/docker/Dockerfile.python3.11 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure python --config=/usr/local/bin/python3-config \ && make -j $NCPU python3-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure python --config=/usr/local/bin/python3-config \ && make -j $NCPU python3-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Dockerfile.ruby3.2 b/pkg/docker/Dockerfile.ruby3.2 index 19a763f7..4a6b60e4 100644 --- a/pkg/docker/Dockerfile.ruby3.2 +++ b/pkg/docker/Dockerfile.ruby3.2 @@ -6,42 +6,46 @@ LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" -LABEL org.opencontainers.image.version="1.29.1" +LABEL org.opencontainers.image.version="1.30.0" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ - && hg clone -u 1.29.1-1 https://hg.nginx.org/unit \ + && hg clone -u 1.30.0-1 https://hg.nginx.org/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure ruby \ && make -j $NCPU ruby-install \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure ruby \ && make -j $NCPU ruby-install \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/Makefile b/pkg/docker/Makefile index 5b1f4f20..cb801253 100644 --- a/pkg/docker/Makefile +++ b/pkg/docker/Makefile @@ -8,90 +8,88 @@ DEFAULT_VERSION := $(NXT_VERSION) VERSION ?= $(DEFAULT_VERSION) PATCHLEVEL ?= 1 -EXPORT_DIR := $(VERSION) - -MODULES ?= go jsc node perl php python ruby minimal +MODULES ?= go jsc node perl php python ruby VARIANT ?= bullseye -VERSION_minimal ?= +VERSIONS_minimal ?= CONTAINER_minimal ?= debian:$(VARIANT)-slim CONFIGURE_minimal ?= INSTALL_minimal ?= version RUN_minimal ?= /bin/true -VERSION_go ?= 1.20 +VERSIONS_go ?= 1.20 VARIANT_go ?= $(VARIANT) -CONTAINER_go ?= golang:$(VERSION_go)-$(VARIANT_go) +$(foreach goversion, $(VERSIONS_go), $(eval CONTAINER_go$(goversion) = golang:$(goversion)-$(VARIANT_go))) CONFIGURE_go ?= go --go-path=$$GOPATH INSTALL_go ?= go-install-src libunit-install RUN_go ?= /bin/true -VERSION_jsc ?= 11 +VERSIONS_jsc ?= 11 VARIANT_jsc ?= jammy -CONTAINER_jsc ?= eclipse-temurin:$(VERSION_jsc)-jdk-$(VARIANT_jsc) +$(foreach jscversion, $(VERSIONS_jsc), $(eval CONTAINER_jsc$(jscversion) = eclipse-temurin:$(jscversion)-jdk-$(VARIANT_jsc))) CONFIGURE_jsc ?= java --jars=/usr/share/unit-jsc-common/ INSTALL_jsc ?= java-shared-install java-install RUN_jsc ?= /bin/true -VERSION_node ?= 18 +VERSIONS_node ?= 18 VARIANT_node ?= $(VARIANT) -CONTAINER_node ?= node:$(VERSION_node)-$(VARIANT_node) +$(foreach nodeversion, $(VERSIONS_node), $(eval CONTAINER_node$(nodeversion) = node:$(nodeversion)-$(VARIANT_node))) CONFIGURE_node ?= nodejs --node-gyp=/usr/local/lib/node_modules/npm/bin/node-gyp-bin/node-gyp INSTALL_node ?= node node-install libunit-install RUN_node ?= /bin/true -VERSION_perl ?= 5.36 +VERSIONS_perl ?= 5.36 VARIANT_perl ?= $(VARIANT) -CONTAINER_perl ?= perl:$(VERSION_perl)-$(VARIANT_perl) +$(foreach perlversion, $(VERSIONS_perl), $(eval CONTAINER_perl$(perlversion) = perl:$(perlversion)-$(VARIANT_perl))) CONFIGURE_perl ?= perl INSTALL_perl ?= perl-install RUN_perl ?= /bin/true -VERSION_php ?= 8.2 +VERSIONS_php ?= 8.2 VARIANT_php ?= cli-$(VARIANT) -CONTAINER_php ?= php:$(VERSION_php)-$(VARIANT_php) +$(foreach phpversion, $(VERSIONS_php), $(eval CONTAINER_php$(phpversion) = php:$(phpversion)-$(VARIANT_php))) CONFIGURE_php ?= php INSTALL_php ?= php-install RUN_php ?= ldconfig -VERSION_python ?= 3.11 +VERSIONS_python ?= 3.11 VARIANT_python ?= $(VARIANT) -CONTAINER_python ?= python:$(VERSION_python)-$(VARIANT_python) +$(foreach pythonversion, $(VERSIONS_python), $(eval CONTAINER_python$(pythonversion) = python:$(pythonversion)-$(VARIANT_python))) CONFIGURE_python ?= python --config=/usr/local/bin/python3-config INSTALL_python ?= python3-install RUN_python ?= /bin/true -VERSION_ruby ?= 3.2 +VERSIONS_ruby ?= 3.2 VARIANT_ruby ?= $(VARIANT) -CONTAINER_ruby ?= ruby:$(VERSION_ruby)-$(VARIANT_ruby) +$(foreach rubyversion, $(VERSIONS_ruby), $(eval CONTAINER_ruby$(rubyversion) = ruby:$(rubyversion)-$(VARIANT_ruby))) CONFIGURE_ruby ?= ruby INSTALL_ruby ?= ruby-install RUN_ruby ?= gem install rack default: - @echo "valid targets: all build dockerfiles clean" + @echo "valid targets: all build dockerfiles library clean" -MODVERSIONS = $(foreach module,$(MODULES),$(module)$(VERSION_$(module))) +MODVERSIONS = $(foreach module, $(MODULES), $(foreach modversion, $(shell for v in $(VERSIONS_$(module)); do echo $$v; done | sort -r), $(module)$(modversion))) minimal modname = $(shell echo $1 | /usr/bin/tr -d '.01234567890-') dockerfiles: $(addprefix Dockerfile., $(MODVERSIONS)) -build: $(addprefix build-,$(MODVERSIONS)) +build: $(addprefix build-, $(MODVERSIONS)) Dockerfile.%: ../../version template.Dockerfile @echo "===> Building $@" cat template.Dockerfile | sed \ -e 's,@@VERSION@@,$(VERSION),g' \ -e 's,@@PATCHLEVEL@@,$(PATCHLEVEL),g' \ - -e 's,@@CONTAINER@@,$(CONTAINER_$(call modname, $*)),g' \ + -e 's,@@CONTAINER@@,$(CONTAINER_$*),g' \ -e 's,@@CONFIGURE@@,$(CONFIGURE_$(call modname, $*)),g' \ -e 's,@@INSTALL@@,$(INSTALL_$(call modname, $*)),g' \ -e 's,@@RUN@@,$(RUN_$(call modname, $*)),g' \ > $@ build-%: Dockerfile.% - docker pull $(CONTAINER_$(call modname, $*)) + docker pull $(CONTAINER_$*) docker build --no-cache -t unit:$(VERSION)-$* -f Dockerfile.$* . library: @@ -99,22 +97,32 @@ library: @echo "" @echo "Maintainers: Unit Docker Maintainers <docker-maint@nginx.com> (@nginx)" @echo "GitRepo: https://github.com/nginx/unit.git" - @for mod in $(MODVERSIONS); do \ + @previous=""; \ + for mod in $(MODVERSIONS); do \ echo ""; \ - TAGS="$$mod $${mod%%.*} $$( echo $$mod | tr -d '.0123456789-' )"; \ + modname="$$( echo $$mod | tr -d '.0123456789-' )"; \ + TAGS="$$mod $${mod%%.*} $$modname" ; \ TAGS="$$(echo $$TAGS | tr " " "\n" | sort -u -r | tr "\n" "," | sed "s/,/, /g")"; \ - echo "Tags: $(VERSION)-$$mod, $${TAGS%, }"; \ + if [ "$$previous" == "$$modname" ]; then \ + echo "Tags: $(VERSION)-$$mod, $$mod"; \ + else \ + if [ "$$mod" == "minimal" ]; then \ + echo "Tags: $(VERSION)-$$mod, $${TAGS%, }, latest"; \ + else \ + echo "Tags: $(VERSION)-$$mod, $${TAGS%, }"; \ + fi; \ + fi; \ echo "Architectures: amd64, arm64v8"; \ echo "GitFetch: refs/heads/branches/packaging"; \ echo "GitCommit: $(shell git describe --always --abbrev=0 HEAD)"; \ echo "Directory: pkg/docker"; \ echo "File: Dockerfile.$$mod"; \ + previous=$$(echo $$mod | tr -d '.0123456789-'); \ done all: $(addprefix Dockerfile., $(MODVERSIONS)) clean: - rm -f $(addprefix Dockerfile., $(MODVERSIONS)) - rm -rf $(EXPORT_DIR) + rm -f Dockerfile.* .PHONY: default build dockerfiles clean library diff --git a/pkg/docker/docker-entrypoint.sh b/pkg/docker/docker-entrypoint.sh index e0afd7ea..4646409f 100755 --- a/pkg/docker/docker-entrypoint.sh +++ b/pkg/docker/docker-entrypoint.sh @@ -25,21 +25,23 @@ if [ "$1" = "unitd" ] || [ "$1" = "unitd-debug" ]; then if /usr/bin/find "/var/lib/unit/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then echo "$0: /var/lib/unit/ is not empty, skipping initial configuration..." else - if /usr/bin/find "/docker-entrypoint.d/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then - echo "$0: /docker-entrypoint.d/ is not empty, launching Unit daemon to perform initial configuration..." - /usr/sbin/$1 --control unix:/var/run/control.unit.sock + echo "$0: Launching Unit daemon to perform initial configuration..." + /usr/sbin/$1 --control unix:/var/run/control.unit.sock - for i in $(/usr/bin/seq $WAITLOOPS); do - if [ ! -S /var/run/control.unit.sock ]; then - echo "$0: Waiting for control socket to be created..." - /bin/sleep $SLEEPSEC - else - break - fi - done - # even when the control socket exists, it does not mean unit has finished initialisation - # this curl call will get a reply once unit is fully launched - /usr/bin/curl -s -X GET --unix-socket /var/run/control.unit.sock http://localhost/ + for i in $(/usr/bin/seq $WAITLOOPS); do + if [ ! -S /var/run/control.unit.sock ]; then + echo "$0: Waiting for control socket to be created..." + /bin/sleep $SLEEPSEC + else + break + fi + done + # even when the control socket exists, it does not mean unit has finished initialisation + # this curl call will get a reply once unit is fully launched + /usr/bin/curl -s -X GET --unix-socket /var/run/control.unit.sock http://localhost/ + + if /usr/bin/find "/docker-entrypoint.d/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then + echo "$0: /docker-entrypoint.d/ is not empty, applying initial configuration..." echo "$0: Looking for certificate bundles in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.pem"); do @@ -47,6 +49,12 @@ if [ "$1" = "unitd" ] || [ "$1" = "unitd-debug" ]; then curl_put $f "certificates/$(basename $f .pem)" done + echo "$0: Looking for JavaScript modules in /docker-entrypoint.d/..." + for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.js"); do + echo "$0: Uploading JavaScript module: $f" + curl_put $f "js_modules/$(basename $f .js)" + done + echo "$0: Looking for configuration snippets in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.json"); do echo "$0: Applying configuration $f"; @@ -60,32 +68,33 @@ if [ "$1" = "unitd" ] || [ "$1" = "unitd-debug" ]; then done # warn on filetypes we don't know what to do with - for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -not -name "*.sh" -not -name "*.json" -not -name "*.pem"); do + for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -not -name "*.sh" -not -name "*.json" -not -name "*.pem" -not -name "*.js"); do echo "$0: Ignoring $f"; done + else + echo "$0: /docker-entrypoint.d/ is empty, creating 'welcome' configuration..." + curl_put /usr/share/unit/welcome/welcome.json "config" + fi - echo "$0: Stopping Unit daemon after initial configuration..." - kill -TERM $(/bin/cat /var/run/unit.pid) + echo "$0: Stopping Unit daemon after initial configuration..." + kill -TERM $(/bin/cat /var/run/unit.pid) - for i in $(/usr/bin/seq $WAITLOOPS); do - if [ -S /var/run/control.unit.sock ]; then - echo "$0: Waiting for control socket to be removed..." - /bin/sleep $SLEEPSEC - else - break - fi - done + for i in $(/usr/bin/seq $WAITLOOPS); do if [ -S /var/run/control.unit.sock ]; then - kill -KILL $(/bin/cat /var/run/unit.pid) - rm -f /var/run/control.unit.sock + echo "$0: Waiting for control socket to be removed..." + /bin/sleep $SLEEPSEC + else + break fi - - echo - echo "$0: Unit initial configuration complete; ready for start up..." - echo - else - echo "$0: /docker-entrypoint.d/ is empty, skipping initial configuration..." + done + if [ -S /var/run/control.unit.sock ]; then + kill -KILL $(/bin/cat /var/run/unit.pid) + rm -f /var/run/control.unit.sock fi + + echo + echo "$0: Unit initial configuration complete; ready for start up..." + echo fi fi diff --git a/pkg/docker/template.Dockerfile b/pkg/docker/template.Dockerfile index 7b749a1a..c6a72aa8 100644 --- a/pkg/docker/template.Dockerfile +++ b/pkg/docker/template.Dockerfile @@ -11,7 +11,7 @@ LABEL org.opencontainers.image.version="@@VERSION@@" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ - && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev \ + && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates mercurial build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ && hg clone -u @@VERSION@@-@@PATCHLEVEL@@ https://hg.nginx.org/unit \ && cd unit \ @@ -19,29 +19,33 @@ RUN set -ex \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ - && CONFIGURE_ARGS="--prefix=/usr \ - --state=/var/lib/unit \ + && CONFIGURE_ARGS_MODULES="--prefix=/usr \ + --statedir=/var/lib/unit \ --control=unix:/var/run/control.unit.sock \ --pid=/var/run/unit.pid \ --log=/var/log/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ + --njs" \ + && make -j $NCPU -C pkg/contrib .njs \ + && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd-debug \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ - && install -pm755 build/unitd /usr/sbin/unitd \ + && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/debug-modules --debug \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure @@CONFIGURE@@ \ && make -j $NCPU @@INSTALL@@ \ && make clean \ - && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --modules=/usr/lib/unit/modules \ + && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure @@CONFIGURE@@ \ && make -j $NCPU @@INSTALL@@ \ && cd \ @@ -71,9 +75,10 @@ RUN set -ex \ && ln -sf /dev/stdout /var/log/unit.log COPY docker-entrypoint.sh /usr/local/bin/ +COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - +EXPOSE 80 CMD ["unitd", "--no-daemon", "--control", "unix:/var/run/control.unit.sock"] diff --git a/pkg/docker/welcome.html b/pkg/docker/welcome.html new file mode 100644 index 00000000..9c4f8281 --- /dev/null +++ b/pkg/docker/welcome.html @@ -0,0 +1,45 @@ +<!DOCTYPE html> +<html> + <head> + <title>Welcome to NGINX Unit</title> + <style type="text/css"> + body { background: white; color: black; font-family: sans-serif; margin: 2em; line-height: 1.5; } + h1,h2 { color: #00974d; } + li { margin-bottom: 0.5em; } + pre { background-color: beige; padding: 0.4em; } + hr { margin-top: 2em; border: 1px solid #00974d; } + .indent { margin-left: 1.5em; } + </style> + </head> + <body> + <h1>Welcome to NGINX Unit</h1> + <p>Congratulations! NGINX Unit is installed and running.</p> + <h3>Useful Links</h3> + <ul> + <li><b><a href="https://unit.nginx.org/configuration/?referer=welcome&platform=docker">https://unit.nginx.org/configuration/</a></b><br> + To get started with Unit, see the <em>Configuration</em> docs, starting with + the <em>Quick Start</em> guide.</li> + <li><b><a href="https://unit.nginx.org/howto/docker/?referer=welcome&platform=docker">https://unit.nginx.org/howto/docker/</a></b><br> + For guidance about running <em>Unit in Docker</em> and tips for containerized + applications. + <li><b><a href="https://github.com/nginx/unit">https://github.com/nginx/unit</a></b><br> + See our GitHub repo to browse the code, contribute, or seek help from the + <a href="https://github.com/nginx/unit#community">community</a>.</li> + </ul> + + <h2>Next steps</h2> + + <h3>Check Current Configuration</h3> + <div class="indent"> + <p>Unit's control API is currently listening for configuration changes + on the <a href="https://en.wikipedia.org/wiki/Unix_domain_socket">Unix socket</a> at + <b>/var/run/control.unit.sock</b> inside the container.<br> + To see the current configuration run:</p> + <pre>docker exec -ti <containerID> curl --unix-socket /var/run/control.unit.sock http://localhost/config</pre> + </div> + + <hr> + <p><a href="https://unit.nginx.org/?referer=welcome&platform=docker">NGINX Unit — the universal web app server</a><br> + NGINX, Inc. © 2023</p> + </body> +</html> diff --git a/pkg/docker/welcome.json b/pkg/docker/welcome.json new file mode 100644 index 00000000..2a148da8 --- /dev/null +++ b/pkg/docker/welcome.json @@ -0,0 +1,25 @@ +{ + "listeners": { + "*:80": { + "pass": "routes" + } + }, + + "routes": [ + { + "match": { + "headers": { + "accept": "*text/html*" + } + }, + "action": { + "share": "/usr/share/unit/welcome/welcome.html" + } + }, + { + "action": { + "share": "/usr/share/unit/welcome/welcome.md" + } + } + ] +} diff --git a/pkg/docker/welcome.md b/pkg/docker/welcome.md new file mode 100644 index 00000000..fef3d152 --- /dev/null +++ b/pkg/docker/welcome.md @@ -0,0 +1,29 @@ +Welcome to NGINX Unit +===================== + +Congratulations! NGINX Unit is installed and running. + +Useful Links +------------ + + * https://unit.nginx.org/ + - Get started with the 'Configuration' docs, starting with the 'Quick Start' guide. + + * https://unit.nginx.org/howto/docker/ + - Guidance for running Unit in a container and tips for containerized applications. + + * https://github.com/nginx/unit + - See our GitHub repo to browse the code, contribute, or seek help from the community. + +Current Configuration +--------------------- +Unit's control API is currently listening for configuration changes on the Unix socket at +`/var/run/control.unit.sock` inside the container. + +Read the current configuration with +``` +docker exec -ti <containerID> curl --unix-socket /var/run/control.unit.sock http://localhost/config +``` + +--- +NGINX Unit - the universal web app server diff --git a/pkg/rpm/Makefile b/pkg/rpm/Makefile index 5c104ca3..d3cc34bd 100644 --- a/pkg/rpm/Makefile +++ b/pkg/rpm/Makefile @@ -18,6 +18,8 @@ else ifeq ($(shell rpm --eval "%{?rhel}"), 9) OSVER = centos9 else ifeq ($(shell rpm --eval "%{?amzn}"), 2) OSVER = amazonlinux2 +else ifeq ($(shell rpm --eval "%{?amzn}"), 2023) +OSVER = amazonlinux2023 else ifeq ($(shell test `rpm --eval '0%{?fedora} -ge 35 -a 0%{?fedora} -le 36'`; echo $$?),0) OSVER = fedora else ifeq ($(shell test `rpm --eval '0%{?fedora} -ge 37'`; echo $$?),0) @@ -84,6 +86,16 @@ include Makefile.jsc-common include Makefile.jsc8 endif +ifeq ($(OSVER), amazonlinux2023) +include Makefile.php +include Makefile.python39 +include Makefile.python311 +include Makefile.go +include Makefile.perl +include Makefile.jsc-common +include Makefile.jsc17 +endif + ifeq ($(OSVER), fedora) include Makefile.php include Makefile.python310 @@ -108,11 +120,11 @@ endif CONFIGURE_ARGS_COMMON=\ --prefix=/usr \ - --state=%{_sharedstatedir}/unit \ + --statedir=%{_sharedstatedir}/unit \ --control="unix:/var/run/unit/control.sock" \ --pid=/var/run/unit/unit.pid \ --log=/var/log/unit/unit.log \ - --tmp=/var/tmp \ + --tmpdir=/var/tmp \ --user=unit \ --group=unit \ --tests \ @@ -142,7 +154,7 @@ check-build-depends-%: esac ; \ not_installed= ; \ for pkg in $${pkgs}; do \ - rpm -qi $${pkg} >/dev/null 2>&1 ; \ + rpm -qi --whatprovides $${pkg} >/dev/null 2>&1 ; \ if [ $$? -ne 0 ]; then \ not_installed="$${not_installed} $${pkg}" ; \ fi ; \ @@ -180,7 +192,8 @@ endif rpmbuild/SOURCES/unit-$(VERSION).tar.gz: cd ../.. && tar -czf pkg/rpm/rpmbuild/SOURCES/unit-$(VERSION).tar.gz \ --transform "s#^#unit-$(VERSION)/#" \ - LICENSE NOTICE CHANGES README.md CONTRIBUTING.md configure auto src test version go pkg/contrib docs/man/unitd.8.in + LICENSE NOTICE CHANGES README.md CONTRIBUTING.md configure auto src \ + test tools version go pkg/contrib docs/man/man8/unitd.8.in unit: check-build-depends-unit rpmbuild/SPECS/unit.spec rpmbuild/SOURCES/unit-$(VERSION).tar.gz @echo "===> Building $@ package" ; \ diff --git a/pkg/rpm/Makefile.jsc-common b/pkg/rpm/Makefile.jsc-common index 9688e265..a3c3a3da 100644 --- a/pkg/rpm/Makefile.jsc-common +++ b/pkg/rpm/Makefile.jsc-common @@ -8,13 +8,21 @@ MODULE_RELEASE_jsc_common= 1 JAVA_ARCH_jsc_common= $(shell /usr/lib/jvm/java-1.8.0/bin/java -XshowSettings 2>&1 | grep -F -e os.arch | sed -e 's/^.*= //') +ifeq ($(OSVER),amazonlinux2023) +MODULE_CONFARGS_jsc_common= java --home=/usr/lib/jvm/java-17-amazon-corretto --lib-path=/usr/lib/jvm/java-17-amazon-corretto/lib --jars=/usr/share/unit-jsc-common/ +else MODULE_CONFARGS_jsc_common= java --home=/usr/lib/jvm/java-1.8.0 --lib-path=/usr/lib/jvm/jre-1.8.0/lib/$(JAVA_ARCH_jsc_common) --jars=/usr/share/unit-jsc-common/ +endif MODULE_MAKEARGS_jsc_common= java MODULE_INSTARGS_jsc_common= java-shared-install MODULE_SOURCES_jsc_common= COPYRIGHT.unit-jsc-common +ifeq ($(OSVER),amazonlinux2023) +BUILD_DEPENDS_jsc_common= java-17-amazon-corretto-devel curl +else BUILD_DEPENDS_jsc_common= java-1.8.0-openjdk-devel curl +endif BUILD_DEPENDS+= $(BUILD_DEPENDS_jsc_common) define MODULE_DEFINITIONS_jsc_common diff --git a/pkg/rpm/Makefile.jsc17 b/pkg/rpm/Makefile.jsc17 new file mode 100644 index 00000000..7efdafaa --- /dev/null +++ b/pkg/rpm/Makefile.jsc17 @@ -0,0 +1,70 @@ +MODULES+= jsc17 +MODULE_SUFFIX_jsc17= jsc17 + +MODULE_SUMMARY_jsc17= Java 17 module for NGINX Unit + +MODULE_VERSION_jsc17= $(VERSION) +MODULE_RELEASE_jsc17= 1 + +MODULE_CONFARGS_jsc17= java --module=java17 --home=/usr/lib/jvm/java-17-amazon-corretto --lib-path=/usr/lib/jvm/java-17-amazon-corretto/lib --jars=/usr/share/unit-jsc-common/ +MODULE_MAKEARGS_jsc17= java17 +MODULE_INSTARGS_jsc17= java17-install + +MODULE_SOURCES_jsc17= unit.example-jsc-app \ + unit.example-jsc17-config + +BUILD_DEPENDS_jsc17= java-17-amazon-corretto-devel +BUILD_DEPENDS+= $(BUILD_DEPENDS_jsc17) + +define MODULE_DEFINITIONS_jsc17 +Requires: unit-jsc-common == $(MODULE_VERSION_jsc_common)-$(MODULE_RELEASE_jsc_common)%{?dist}.ngx +Requires: java-17-amazon-corretto-headless +endef +export MODULE_DEFINITIONS_jsc17 + +define MODULE_PREINSTALL_jsc17 +%{__mkdir} -p %{buildroot}%{_datadir}/doc/unit-jsc17/examples/jsc-app +%{__install} -m 644 -p %{SOURCE100} \ + %{buildroot}%{_datadir}/doc/unit-jsc17/examples/jsc-app/index.jsp +%{__install} -m 644 -p %{SOURCE101} \ + %{buildroot}%{_datadir}/doc/unit-jsc17/examples/unit.config +%{__install} -m 644 -p %{bdir}/src/java/README.JSR-340 \ + %{buildroot}%{_datadir}/doc/unit-jsc17/ +endef +export MODULE_PREINSTALL_jsc17 + +define MODULE_POSTINSTALL_jsc17 +DESTDIR=%{buildroot} make java-shared-uninstall +endef +export MODULE_POSTINSTALL_jsc17 + +define MODULE_FILES_jsc17 +%{_libdir}/unit/modules/* +%{_libdir}/unit/debug-modules/* +%dir %{_datadir}/doc/unit-jsc17 +%{_datadir}/doc/unit-jsc17/* +%{_datadir}/unit-jsc-common/* +endef +export MODULE_FILES_jsc17 + +define MODULE_POST_jsc17 +cat <<BANNER +---------------------------------------------------------------------- + +The $(MODULE_SUMMARY_jsc17) has been installed. + +To check out the sample app, run these commands: + + sudo service unit restart + cd /usr/share/doc/%{name}/examples + sudo curl -X PUT --data-binary @unit.config --unix-socket /var/run/unit/control.sock http://localhost/config + curl http://localhost:8800/ + +Online documentation is available at https://unit.nginx.org + +`cat /usr/share/doc/unit-jsc17/README.JSR-340` + +---------------------------------------------------------------------- +BANNER +endef +export MODULE_POST_jsc17 diff --git a/pkg/rpm/Makefile.python311 b/pkg/rpm/Makefile.python311 index a8bee943..ae58d722 100644 --- a/pkg/rpm/Makefile.python311 +++ b/pkg/rpm/Makefile.python311 @@ -15,6 +15,8 @@ MODULE_SOURCES_python311= unit.example-python-app \ ifneq (,$(findstring $(OSVER),fedora37)) BUILD_DEPENDS_python311= python3-devel +else +BUILD_DEPENDS_python311= python3.11-devel endif BUILD_DEPENDS+= $(BUILD_DEPENDS_python311) diff --git a/pkg/rpm/Makefile.python39 b/pkg/rpm/Makefile.python39 index 5d7c2327..7a3ae0b0 100644 --- a/pkg/rpm/Makefile.python39 +++ b/pkg/rpm/Makefile.python39 @@ -13,7 +13,7 @@ MODULE_INSTARGS_python39= python3.9-install MODULE_SOURCES_python39= unit.example-python-app \ unit.example-python39-config -ifneq (,$(findstring $(OSVER),fedora amazonlinux2 centos9)) +ifneq (,$(findstring $(OSVER),fedora amazonlinux2 amazonlinux2023 centos9)) BUILD_DEPENDS_python39= python3-devel else BUILD_DEPENDS_python39= python39-devel diff --git a/pkg/rpm/rpmbuild/SOURCES/unit.example-jsc17-config b/pkg/rpm/rpmbuild/SOURCES/unit.example-jsc17-config new file mode 100644 index 00000000..28b13e4d --- /dev/null +++ b/pkg/rpm/rpmbuild/SOURCES/unit.example-jsc17-config @@ -0,0 +1,15 @@ +{ + "applications": { + "example_java17": { + "processes": 1, + "type": "java 17", + "webapp": "/usr/share/doc/unit-jsc17/examples/jsc-app" + } + }, + + "listeners": { + "*:8800": { + "pass": "applications/example_java17" + } + } +} diff --git a/pkg/rpm/unit.module.spec.in b/pkg/rpm/unit.module.spec.in index bc68a254..04323afc 100644 --- a/pkg/rpm/unit.module.spec.in +++ b/pkg/rpm/unit.module.spec.in @@ -61,7 +61,7 @@ tar --strip-components=1 -zxf %{SOURCE0} %build ./configure \ %{CONFIGURE_ARGS} \ - --modules=%{_libdir}/unit/debug-modules \ + --modulesdir=%{_libdir}/unit/debug-modules \ --cc-opt="%{CC_OPT}" \ --debug ./configure %%MODULE_CONFARGS%% @@ -69,7 +69,7 @@ make %%MODULE_MAKEARGS%% %{__mv} build build-debug ./configure \ %{CONFIGURE_ARGS} \ - --modules=%{_libdir}/unit/modules \ + --modulesdir=%{_libdir}/unit/modules \ --cc-opt="%{CC_OPT}" ./configure %%MODULE_CONFARGS%% make %%MODULE_MAKEARGS%% diff --git a/pkg/rpm/unit.spec.in b/pkg/rpm/unit.spec.in index 06880fcf..14a2ea00 100644 --- a/pkg/rpm/unit.spec.in +++ b/pkg/rpm/unit.spec.in @@ -80,19 +80,19 @@ Library and include files required for NGINX Unit modules development. PKG_CONFIG_PATH=%{bdir}/pkg/contrib/njs/build \ ./configure \ %{CONFIGURE_ARGS} \ - --modules=%{_libdir}/unit/debug-modules \ + --modulesdir=%{_libdir}/unit/debug-modules \ --libdir=%{_libdir} \ --cc-opt="%{CC_OPT}" \ --ld-opt="%{LD_OPT}" \ --debug %{__make} %{?_smp_mflags} -%{__make} %{?_smp_mflags} build/libunit.a +%{__make} %{?_smp_mflags} build/lib/libunit.a %{__mv} build build-debug PKG_CONFIG_PATH=%{bdir}/pkg/contrib/njs/build \ ./configure \ %{CONFIGURE_ARGS} \ - --modules=%{_libdir}/unit/modules \ + --modulesdir=%{_libdir}/unit/modules \ --libdir=%{_libdir} \ --cc-opt="%{CC_OPT}" \ --ld-opt="%{LD_OPT}" @@ -103,9 +103,14 @@ PKG_CONFIG_PATH=%{bdir}/pkg/contrib/njs/build \ %{__rm} -rf %{buildroot} %{__ln_s} build-nodebug build DESTDIR=%{buildroot} make unitd-install libunit-install manpage-install -%{__install} -m755 %{bdir}/build-debug/unitd \ +%{__mkdir} -p %{buildroot}%{_bindir} +%{__install} -m755 %{bdir}/tools/unitc \ + %{buildroot}%{_bindir}/unitc +%{__install} -m755 %{bdir}/tools/setup-unit \ + %{buildroot}%{_bindir}/setup-unit +%{__install} -m755 %{bdir}/build-debug/sbin/unitd \ %{buildroot}%{_sbindir}/unitd-debug -%{__install} -m644 %{bdir}/build-debug/libunit.a \ +%{__install} -m644 %{bdir}/build-debug/lib/libunit.a \ %{buildroot}%{_libdir}/libunit-debug.a %{__mkdir} -p %{buildroot}%{_libdir}/unit/modules %{__mkdir} -p %{buildroot}%{_libdir}/unit/debug-modules @@ -197,6 +202,8 @@ BANNER %files %defattr(-,root,root,-) +%attr(0755,root,root) %{_bindir}/unitc +%attr(0755,root,root) %{_bindir}/setup-unit %attr(0755,root,root) %{_sbindir}/unitd %attr(0755,root,root) %{_sbindir}/unitd-debug %{_unitdir}/unit.service diff --git a/src/nodejs/unit-http/binding.gyp b/src/nodejs/unit-http/binding.gyp index 55d965bd..e41db7c4 100644 --- a/src/nodejs/unit-http/binding.gyp +++ b/src/nodejs/unit-http/binding.gyp @@ -12,7 +12,7 @@ ], 'sources': ["unit.cpp", "addon.cpp"], 'include_dirs': [ - "<!(echo $UNIT_SRC_PATH)", "<!(echo $UNIT_BUILD_PATH)" + "<!(echo $UNIT_SRC_PATH)", "<!(echo $UNIT_BUILD_PATH/include)" ], 'libraries': [ "<!(echo $UNIT_LIB_STATIC_PATH)" diff --git a/src/nxt_application.c b/src/nxt_application.c index 786c768b..ffa8eb53 100644 --- a/src/nxt_application.c +++ b/src/nxt_application.c @@ -933,6 +933,59 @@ nxt_app_set_environment(nxt_conf_value_t *environment) } +nxt_int_t +nxt_app_set_logs(void) +{ + nxt_int_t ret; + nxt_file_t file; + nxt_task_t *task; + nxt_thread_t *thr; + nxt_process_t *process; + nxt_runtime_t *rt; + nxt_common_app_conf_t *app_conf; + + thr = nxt_thread(); + + task = thr->task; + + rt = task->thread->runtime; + if (!rt->daemon) { + return NXT_OK; + } + + process = rt->port_by_type[NXT_PROCESS_PROTOTYPE]->process; + app_conf = process->data.app; + + if (app_conf->stdout_log != NULL) { + nxt_memzero(&file, sizeof(nxt_file_t)); + file.log_level = 1; + file.name = (u_char *) app_conf->stdout_log; + ret = nxt_file_open(task, &file, O_WRONLY | O_APPEND, O_CREAT, 0666); + if (ret == NXT_ERROR) { + return NXT_ERROR; + } + + nxt_file_stdout(&file); + nxt_file_close(task, &file); + } + + if (app_conf->stderr_log != NULL) { + nxt_memzero(&file, sizeof(nxt_file_t)); + file.log_level = 1; + file.name = (u_char *) app_conf->stderr_log; + ret = nxt_file_open(task, &file, O_WRONLY | O_APPEND, O_CREAT, 0666); + if (ret == NXT_ERROR) { + return NXT_ERROR; + } + + nxt_file_stderr(&file); + nxt_file_close(task, &file); + } + + return NXT_OK; +} + + static u_char * nxt_cstr_dup(nxt_mp_t *mp, u_char *dst, u_char *src) { diff --git a/src/nxt_application.h b/src/nxt_application.h index 4d624448..2675e6a0 100644 --- a/src/nxt_application.h +++ b/src/nxt_application.h @@ -92,6 +92,9 @@ struct nxt_common_app_conf_s { nxt_str_t user; nxt_str_t group; + char *stdout_log; + char *stderr_log; + char *working_directory; nxt_conf_value_t *environment; @@ -141,5 +144,6 @@ extern nxt_app_module_t nxt_external_module; NXT_EXPORT nxt_int_t nxt_unit_default_init(nxt_task_t *task, nxt_unit_init_t *init, nxt_common_app_conf_t *conf); +NXT_EXPORT nxt_int_t nxt_app_set_logs(void); #endif /* _NXT_APPLICATION_H_INCLIDED_ */ diff --git a/src/nxt_buf_filter.c b/src/nxt_buf_filter.c deleted file mode 100644 index 83e5baa9..00000000 --- a/src/nxt_buf_filter.c +++ /dev/null @@ -1,449 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -static nxt_int_t nxt_buf_filter_nobuf(nxt_buf_filter_t *f); -nxt_inline void nxt_buf_filter_next(nxt_buf_filter_t *f); -static void nxt_buf_filter_file_read_start(nxt_task_t *task, - nxt_buf_filter_t *f); -static void nxt_buf_filter_file_read(nxt_task_t *task, nxt_buf_filter_t *f); -static void nxt_buf_filter_file_job_completion(nxt_task_t *task, void *obj, - void *data); -static void nxt_buf_filter_buf_completion(nxt_task_t *task, void *obj, - void *data); -static void nxt_buf_filter_file_read_error(nxt_task_t *task, void *obj, - void *data); - - -void -nxt_buf_filter_add(nxt_task_t *task, nxt_buf_filter_t *f, nxt_buf_t *b) -{ - nxt_buf_chain_add(&f->input, b); - - nxt_buf_filter(task, f, NULL); -} - - -void -nxt_buf_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_int_t ret; - nxt_buf_t *b; - nxt_buf_filter_t *f; - - f = obj; - - nxt_debug(task, "buf filter"); - - if (f->done) { - return; - } - - f->queued = 0; - - for ( ;; ) { - /* - * f->input is a chain of original incoming buffers: memory, - * mapped, file, and sync buffers; - * f->current is a currently processed memory buffer or a chain - * of memory/file or mapped/file buffers which are read of - * or populated from file; - * f->output is a chain of output buffers; - * f->last is the last output buffer in the chain. - */ - - b = f->current; - - nxt_debug(task, "buf filter current: %p", b); - - if (b == NULL) { - - if (f->reading) { - return; - } - - b = f->input; - - nxt_debug(task, "buf filter input: %p", b); - - if (b == NULL) { - /* - * The end of the input chain, pass - * the output chain to the next filter. - */ - nxt_buf_filter_next(f); - - return; - } - - if (nxt_buf_is_mem(b)) { - - f->current = b; - f->input = b->next; - b->next = NULL; - - } else if (nxt_buf_is_file(b)) { - - if (f->run->filter_ready(f) != NXT_OK) { - nxt_buf_filter_next(f); - } - - nxt_buf_filter_file_read_start(task, f); - return; - } - } - - if (nxt_buf_is_sync(b)) { - - ret = NXT_OK; - f->current = b; - f->input = b->next; - b->next = NULL; - - if (nxt_buf_is_nobuf(b)) { - ret = f->run->filter_sync_nobuf(f); - - } else if (nxt_buf_is_flush(b)) { - ret = f->run->filter_sync_flush(f); - - } else if (nxt_buf_is_last(b)) { - ret = f->run->filter_sync_last(f); - - f->done = (ret == NXT_OK); - } - - if (nxt_fast_path(ret == NXT_OK)) { - continue; - } - - if (nxt_slow_path(ret == NXT_ERROR)) { - goto fail; - } - - /* ret == NXT_AGAIN: No filter internal buffers available. */ - goto nobuf; - } - - ret = f->run->filter_process(f); - - if (nxt_fast_path(ret == NXT_OK)) { - b = f->current; - /* - * A filter may just move f->current to f->output - * and then set f->current to NULL. - */ - if (b != NULL && b->mem.pos == b->mem.free) { - f->current = b->next; - nxt_thread_work_queue_add(task->thread, f->work_queue, - b->completion_handler, - task, b, b->parent); - } - - continue; - } - - if (nxt_slow_path(ret == NXT_ERROR)) { - goto fail; - } - - /* ret == NXT_AGAIN: No filter internal buffers available. */ - goto nobuf; - } - -nobuf: - - /* ret == NXT_AGAIN: No filter internal buffers available. */ - - if (nxt_buf_filter_nobuf(f) == NXT_OK) { - return; - } - -fail: - - nxt_thread_work_queue_add(task->thread, f->work_queue, f->run->filter_error, - task, f, f->data); -} - - -static nxt_int_t -nxt_buf_filter_nobuf(nxt_buf_filter_t *f) -{ - nxt_buf_t *b; - - nxt_thread_log_debug("buf filter nobuf"); - - b = nxt_buf_sync_alloc(f->mem_pool, NXT_BUF_SYNC_NOBUF); - - if (nxt_fast_path(b != NULL)) { - - nxt_buf_chain_add(&f->output, b); - f->last = NULL; - - f->run->filter_next(f); - - f->output = NULL; - - return NXT_OK; - } - - return NXT_ERROR; -} - - -nxt_inline void -nxt_buf_filter_next(nxt_buf_filter_t *f) -{ - if (f->output != NULL) { - f->last = NULL; - - f->run->filter_next(f); - f->output = NULL; - } -} - - -void -nxt_buf_filter_enqueue(nxt_task_t *task, nxt_buf_filter_t *f) -{ - nxt_debug(task, "buf filter enqueue: %d", f->queued); - - if (!f->queued && !f->done) { - f->queued = 1; - nxt_thread_work_queue_add(task->thread, f->work_queue, nxt_buf_filter, - task, f, NULL); - } -} - - -static void -nxt_buf_filter_file_read_start(nxt_task_t *task, nxt_buf_filter_t *f) -{ - nxt_job_file_t *jbf; - nxt_buf_filter_file_t *ff; - - ff = f->run->job_file_create(f); - - if (nxt_slow_path(ff == NULL)) { - nxt_thread_work_queue_add(task->thread, f->work_queue, - f->run->filter_error, - task, f, f->data); - return; - } - - f->filter_file = ff; - - jbf = &ff->job_file; - jbf->file = *f->input->file; - - jbf->ready_handler = nxt_buf_filter_file_job_completion; - jbf->error_handler = nxt_buf_filter_file_read_error; - - nxt_job_set_name(&jbf->job, "buf filter job file"); - - f->reading = 1; - - nxt_buf_filter_file_read(task, f); -} - - -static void -nxt_buf_filter_file_read(nxt_task_t *task, nxt_buf_filter_t *f) -{ - nxt_int_t ret; - nxt_off_t size; - nxt_buf_t *b; - nxt_buf_filter_file_t *ff; - - ff = f->filter_file; - - if (ff->job_file.buffer != NULL) { - /* File is now being read. */ - return; - } - - size = f->input->file_end - f->input->file_pos; - - if (size > (nxt_off_t) NXT_SIZE_T_MAX) { - /* - * Small size value is a hint for buffer pool allocation - * size, but if size of the size_t type is lesser than size - * of the nxt_off_t type, the large size value may be truncated, - * so use a default buffer pool allocation size. - */ - size = 0; - } - - if (f->mmap) { - ret = nxt_buf_pool_mmap_alloc(&ff->buffers, (size_t) size); - - } else { - ret = nxt_buf_pool_file_alloc(&ff->buffers, (size_t) size); - } - - if (nxt_fast_path(ret == NXT_OK)) { - b = ff->buffers.current; - - b->file_pos = f->input->file_pos; - b->file_end = f->input->file_pos; - b->file = f->input->file; - - ff->job_file.buffer = b; - ff->job_file.offset = f->input->file_pos; - - f->run->job_file_retain(f); - - nxt_job_file_read(task, &ff->job_file.job); - return; - } - - if (nxt_fast_path(ret != NXT_ERROR)) { - - /* ret == NXT_AGAIN: No buffers available. */ - - if (f->buffering) { - f->buffering = 0; - - if (nxt_fast_path(f->run->filter_flush(f) != NXT_ERROR)) { - return; - } - - } else if (nxt_fast_path(nxt_buf_filter_nobuf(f) == NXT_OK)) { - return; - } - } - - nxt_thread_work_queue_add(task->thread, f->work_queue, f->run->filter_error, - task, f, f->data); -} - - -typedef struct { - nxt_buf_filter_t *filter; - nxt_buf_t *buf; -} nxt_buf_filter_ctx_t; - - -static void -nxt_buf_filter_file_job_completion(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b; - nxt_bool_t done; - nxt_job_file_t *jbf; - nxt_buf_filter_t *f; - nxt_buf_filter_ctx_t *ctx; - - jbf = obj; - f = data; - b = jbf->buffer; - jbf->buffer = NULL; - - nxt_debug(task, "buf filter file completion: \"%FN\" %O-%O", - jbf->file.name, b->file_pos, b->file_end); - - f->run->job_file_release(f); - - ctx = nxt_mem_cache_alloc0(f->mem_pool, sizeof(nxt_buf_filter_ctx_t)); - if (nxt_slow_path(ctx == NULL)) { - goto fail; - } - - ctx->filter = f; - ctx->buf = f->input; - - f->input->file_pos = b->file_end; - - done = (f->input->file_pos == f->input->file_end); - - if (done) { - f->input = f->input->next; - f->reading = 0; - } - - b->data = f->data; - b->completion_handler = nxt_buf_filter_buf_completion; - b->parent = (nxt_buf_t *) ctx; - b->next = NULL; - - nxt_buf_chain_add(&f->current, b); - - nxt_buf_filter(task, f, NULL); - - if (b->mem.pos == b->mem.free) { - /* - * The buffer has been completely processed by nxt_buf_filter(), - * its completion handler has been placed in workqueue and - * nxt_buf_filter_buf_completion() should be eventually called. - */ - return; - } - - if (!done) { - /* Try to allocate another buffer and read the next file part. */ - nxt_buf_filter_file_read(task, f); - } - - return; - -fail: - - nxt_thread_work_queue_add(task->thread, f->work_queue, f->run->filter_error, - task, f, f->data); -} - - -static void -nxt_buf_filter_buf_completion(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *fb, *b; - nxt_buf_filter_t *f; - nxt_buf_filter_ctx_t *ctx; - - b = obj; - ctx = data; - f = ctx->filter; - - nxt_debug(task, "buf filter completion: %p \"%FN\" %O-%O", - b, f->filter_file->job_file.file.name, b->file_pos, b->file_end); - - /* nxt_http_send_filter() might clear a buffer's file status. */ - b->is_file = 1; - - fb = ctx->buf; - - nxt_mp_free(f->mem_pool, ctx); - nxt_buf_pool_free(&f->filter_file->buffers, b); - - if (fb->file_pos < fb->file_end) { - nxt_buf_filter_file_read(task, f); - return; - } - - if (b->file_end == fb->file_end) { - nxt_buf_pool_destroy(&f->filter_file->buffers); - - nxt_job_destroy(&f->filter_file->job_file.job); - - nxt_thread_work_queue_add(task->thread, f->work_queue, - fb->completion_handler, - task, fb, fb->parent); - } - - nxt_buf_filter(task, f, NULL); -} - - -static void -nxt_buf_filter_file_read_error(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_filter_t *f; - - f = data; - - nxt_thread_work_queue_add(task->thread, f->work_queue, f->run->filter_error, - task, f, f->data); -} diff --git a/src/nxt_buf_filter.h b/src/nxt_buf_filter.h deleted file mode 100644 index 27487baa..00000000 --- a/src/nxt_buf_filter.h +++ /dev/null @@ -1,115 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_BUF_FILTER_H_INCLUDED_ -#define _NXT_BUF_FILTER_H_INCLUDED_ - - -/* - * nxt_buf_filter is a framework intended to simplify processing file - * buffers content by a filter. The filter should set callbacks and - * call nxt_buf_filter_add() to start processing. - * - * At first buf_filter calls filter_ready() and the filter ensures - * it may allocate or reuse its internal buffer. No real allocation - * is performed at this step. - * - * TODO prevent unneeded allocaiton if no input data. - * - * - * TODO: The filter can flush data buffered - * previously, if all internal buffers are full. - * - * Then buf_filter looks buffer chains. There are two buffer chains: - * the input chain is a chain of original incoming memory, file, and sync - * buffers; and the current chain is a chain of memory/file buffers read - * from a file-only buffer. The current chain is processed first. Since - * buffers in this chain always contains a memory part, they can be passed - * one by one to the filter using filter_process(). If there is an output - * buffer after the buffer processing, it is added to output chain. The - * output buffers are not filter internal buffers. They just point to these - * internal buffers and one internal buffer can correspond to several output - * buffers which point to adjoining parts of the internal buffer. Further - * processing depends on filter_process() result code: if it returns NXT_OK, - * then the filter internal buffer is not full and buf_filter looks the next - * current or input buffer. If result code is NXT_AGAIN, then the filter - * internal buffer is full and buf_filter calls filter_flush() and then - * schedules to run nxt_buf_filter_repeat(). nxt_buf_filter_repeat() will - * run after all ready output buffer completion handlers and will call - * buf_filter again if no one completion handler will do it already using - * nxt_buf_filter_enqueue(). So in any case buf_filter will run again only - * once. - * - * TODO: - * in ideal just one the filter internal buffer. - * This allows to minimize number of the filter internal buffers if they - * flush fast. - * - * If the current chain is empty, the buf_filter processes the input chain. - * Memory buffers are passed to the filter using filter_process(). If an - * input buffer is a file buffer, then buf_filter calls filter_flush() - * and starts a file job to read the buffer in memory. The file job reads - * file parts into memory/file buffers and adds them to the current chain. - * - * Sync buffers are passed to the filter using filter_sync(). Its - * post-processing is similar to the filter_process() post-processing, - * except sync buffers are always added unmodified to the output chain. - */ - -typedef struct { - nxt_job_file_t job_file; - nxt_buf_pool_t buffers; -} nxt_buf_filter_file_t; - - -typedef struct nxt_buf_filter_s nxt_buf_filter_t; - -typedef struct { - nxt_int_t (*filter_ready)(nxt_buf_filter_t *f); - nxt_int_t (*filter_process)(nxt_buf_filter_t *f); - nxt_int_t (*filter_flush)(nxt_buf_filter_t *f); - - nxt_int_t (*filter_sync_nobuf)(nxt_buf_filter_t *f); - nxt_int_t (*filter_sync_flush)(nxt_buf_filter_t *f); - nxt_int_t (*filter_sync_last)(nxt_buf_filter_t *f); - - void (*filter_next)(nxt_buf_filter_t *f); - nxt_work_handler_t filter_error; - - nxt_buf_filter_file_t *(*job_file_create)(nxt_buf_filter_t *f); - void (*job_file_retain)(nxt_buf_filter_t *f); - void (*job_file_release)(nxt_buf_filter_t *f); -} nxt_buf_filter_ops_t; - - -struct nxt_buf_filter_s { - nxt_buf_t *current; - nxt_buf_t *input; - nxt_buf_t *output; - nxt_buf_t *last; - - nxt_work_queue_t *work_queue; - nxt_buf_filter_file_t *filter_file; - void *data; - nxt_mp_t *mem_pool; - - const nxt_buf_filter_ops_t *run; - - uint8_t mmap; /* 1 bit */ - uint8_t done; /* 1 bit */ - uint8_t queued; /* 1 bit */ - uint8_t reading; /* 1 bit */ - uint8_t buffering; /* 1 bit */ -}; - - -NXT_EXPORT void nxt_buf_filter_add(nxt_task_t *task, nxt_buf_filter_t *f, - nxt_buf_t *b); -NXT_EXPORT void nxt_buf_filter(nxt_task_t *task, void *obj, void *data); -NXT_EXPORT void nxt_buf_filter_enqueue(nxt_task_t *task, nxt_buf_filter_t *f); - - -#endif /* _NXT_BUF_FILTER_H_INCLUDED_ */ diff --git a/src/nxt_cache.c b/src/nxt_cache.c deleted file mode 100644 index e81d63dc..00000000 --- a/src/nxt_cache.c +++ /dev/null @@ -1,642 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -/* A cache time resolution is 10ms. */ -#define nxt_cache_time(thr) \ - (uint64_t) (nxt_thread_time(thr) * 100) - - -static nxt_int_t nxt_cache_lvlhsh_test(nxt_lvlhsh_query_t *lhq, void *data); -static nxt_work_handler_t nxt_cache_query_locked(nxt_cache_t *cache, - nxt_cache_query_t *q, nxt_lvlhsh_query_t *lhq); -static nxt_work_handler_t nxt_cache_node_hold(nxt_cache_t *cache, - nxt_cache_query_t *q, nxt_lvlhsh_query_t *lhq); -static nxt_work_handler_t nxt_cache_node_test(nxt_cache_t *cache, - nxt_cache_query_t *q); - -static void nxt_cache_wait_handler(nxt_thread_t *thr, void *obj, void *data); -static void nxt_cache_timeout_handler(nxt_thread_t *thr, void *obj, void *data); -static void nxt_cache_wake_handler(nxt_thread_t *thr, void *obj, void *data); -static ssize_t nxt_cache_release_locked(nxt_cache_t *cache, - nxt_cache_query_t *q, u_char *buf, size_t size); - -static nxt_cache_node_t *nxt_cache_node_alloc(nxt_cache_t *cache); -static void nxt_cache_node_free(nxt_cache_t *cache, nxt_cache_node_t *node, - nxt_bool_t fast); -static nxt_cache_query_wait_t *nxt_cache_query_wait_alloc(nxt_cache_t *cache, - nxt_bool_t *slow); -static void nxt_cache_query_wait_free(nxt_cache_t *cache, - nxt_cache_query_wait_t *qw); - - -/* STUB */ -nxt_int_t nxt_cache_shm_create(nxt_mem_zone_t *pool); -static void *nxt_cache_shm_alloc(void *data, size_t size, nxt_uint_t nalloc); -/**/ - - -nxt_int_t -nxt_cache_shm_create(nxt_mem_zone_t *mz) -{ - nxt_cache_t *cache; - - static const nxt_lvlhsh_proto_t proto nxt_aligned(64) = { - NXT_LVLHSH_LARGE_SLAB, - 0, - nxt_cache_lvlhsh_test, - (nxt_lvlhsh_alloc_t) nxt_cache_shm_alloc, - (nxt_lvlhsh_free_t) nxt_mem_zone_free, - }; - - cache = nxt_mem_zone_zalloc(mz, sizeof(nxt_cache_t)); - - if (cache == NULL) { - return NXT_ERROR; - } - - cache->proto = &proto; - cache->pool = mz; - - cache->start_time = nxt_cache_time(nxt_thread()); - - return NXT_OK; -} - - -static void * -nxt_cache_shm_alloc(void *data, size_t size, nxt_uint_t nalloc) -{ - return nxt_mem_zone_align(data, size, size); -} - - -void -nxt_cache_init(nxt_cache_t *cache) -{ - static const nxt_lvlhsh_proto_t proto nxt_aligned(64) = { - NXT_LVLHSH_LARGE_MEMALIGN, - 0, - nxt_cache_lvlhsh_test, - nxt_lvlhsh_alloc, - nxt_lvlhsh_free, - }; - - cache->proto = &proto; - - cache->start_time = nxt_cache_time(nxt_thread()); -} - - -static nxt_int_t -nxt_cache_lvlhsh_test(nxt_lvlhsh_query_t *lhq, void *data) -{ - nxt_cache_node_t *node; - - node = data; - - if (nxt_str_eq(&lhq->key, node->key_data, node->key_len)) { - return NXT_OK; - } - - return NXT_DECLINED; -} - - -nxt_inline void -nxt_cache_lock(nxt_cache_t *cache) -{ - if (cache->shared) { - nxt_thread_spin_lock(&cache->lock); - } -} - - -nxt_inline void -nxt_cache_unlock(nxt_cache_t *cache) -{ - if (cache->shared) { - nxt_thread_spin_unlock(&cache->lock); - } -} - - -void -nxt_cache_query(nxt_cache_t *cache, nxt_cache_query_t *q) -{ - nxt_thread_t *thr; - nxt_lvlhsh_query_t lhq; - nxt_work_handler_t handler; - - thr = nxt_thread(); - - if (cache != NULL) { - lhq.key_hash = nxt_murmur_hash2(q->key_data, q->key_len); - lhq.replace = 0; - lhq.key.len = q->key_len; - lhq.key.data = q->key_data; - lhq.proto = cache->proto; - lhq.pool = cache->pool; - - q->now = nxt_cache_time(thr); - - nxt_cache_lock(cache); - - handler = nxt_cache_query_locked(cache, q, &lhq); - - nxt_cache_unlock(cache); - - } else { - handler = q->state->nocache_handler; - } - - handler(thr, q, NULL); -} - - -static nxt_work_handler_t -nxt_cache_query_locked(nxt_cache_t *cache, nxt_cache_query_t *q, - nxt_lvlhsh_query_t *lhq) -{ - nxt_int_t ret; - nxt_time_t expiry; - nxt_cache_node_t *node; - nxt_cache_query_state_t *state; - - if (q->hold) { - return nxt_cache_node_hold(cache, q, lhq); - } - - ret = nxt_lvlhsh_find(&cache->lvlhsh, lhq); - - state = q->state; - - if (ret != NXT_OK) { - /* NXT_DECLINED */ - return state->nocache_handler; - } - - node = lhq->value; - node->count++; - q->node = node; - - expiry = cache->start_time + node->expiry; - - if (q->now < expiry) { - return state->ready_handler; - } - - q->stale = 1; - - return state->stale_handler; -} - - -static nxt_work_handler_t -nxt_cache_node_hold(nxt_cache_t *cache, nxt_cache_query_t *q, - nxt_lvlhsh_query_t *lhq) -{ - nxt_int_t ret; - nxt_bool_t slow; - nxt_cache_node_t *node, *sentinel; - nxt_work_handler_t handler; - nxt_cache_query_wait_t *qw; - nxt_cache_query_state_t *state; - - state = q->state; - sentinel = nxt_cache_node_alloc(cache); - - if (nxt_slow_path(sentinel == NULL)) { - return state->error_handler; - } - - sentinel->key_data = q->key_data; - sentinel->key_len = q->key_len; - lhq->value = sentinel; - - /* - * Try to insert an empty sentinel node to hold updating - * process if there is no existent cache node in cache. - */ - ret = nxt_lvlhsh_insert(&cache->lvlhsh, lhq); - - if (ret == NXT_OK) { - /* The sentinel node was successully added. */ - - q->node = sentinel; - sentinel->updating = 1; - return state->update_handler; - } - - nxt_cache_node_free(cache, sentinel, 1); - - if (ret == NXT_ERROR) { - return state->error_handler; - } - - /* NXT_DECLINED: a cache node exists. */ - - node = lhq->value; - node->count++; - q->node = node; - - handler = nxt_cache_node_test(cache, q); - if (handler != NULL) { - return handler; - } - - /* Add the node to a wait queue. */ - - qw = nxt_cache_query_wait_alloc(cache, &slow); - if (nxt_slow_path(qw == NULL)) { - return state->error_handler; - } - - if (slow) { - /* The node state may have been changed during slow allocation. */ - - handler = nxt_cache_node_test(cache, q); - if (handler != NULL) { - nxt_cache_query_wait_free(cache, qw); - return handler; - } - } - - qw->query = q; - qw->next = node->waiting; - qw->busy = 0; - qw->deleted = 0; - qw->pid = nxt_pid; - qw->engine = nxt_thread_event_engine(); - qw->handler = nxt_cache_wake_handler; - qw->cache = cache; - - node->waiting = qw; - - return nxt_cache_wait_handler; -} - - -static nxt_work_handler_t -nxt_cache_node_test(nxt_cache_t *cache, nxt_cache_query_t *q) -{ - nxt_time_t expiry; - nxt_cache_node_t *node; - nxt_cache_query_state_t *state; - - q->stale = 0; - state = q->state; - node = q->node; - - expiry = cache->start_time + node->expiry; - - if (q->now < expiry) { - return state->ready_handler; - } - - /* - * A valid stale or empty sentinel cache node. - * The sentinel node can be only in updating state. - */ - - if (node->updating) { - - if (node->expiry != 0) { - /* A valid stale cache node. */ - - q->stale = 1; - - if (q->use_stale) { - return state->stale_handler; - } - } - - /* A sentinel node. */ - return NULL; - } - - /* A valid stale cache node is not being updated now. */ - - q->stale = 1; - - if (q->use_stale) { - - if (q->update_stale) { - node->updating = 1; - return state->update_stale_handler; - } - - return state->stale_handler; - } - - node->updating = 1; - return state->update_handler; -} - - -static void -nxt_cache_wait_handler(nxt_thread_t *thr, void *obj, void *data) -{ - nxt_event_timer_t *ev; - nxt_cache_query_t *cq; - - cq = obj; - - if (cq->timeout != 0) { - - ev = &cq->timer; - - if (ev->state == NXT_EVENT_TIMER_DISABLED) { - ev->handler = nxt_cache_timeout_handler; - nxt_event_timer_ident(ev, -1); - - nxt_event_timer_add(thr->engine, ev, cq->timeout); - } - } -} - - -static void -nxt_cache_timeout_handler(nxt_thread_t *thr, void *obj, void *data) -{ - nxt_cache_query_t *cq; - nxt_event_timer_t *ev; - - ev = obj; - - cq = nxt_event_timer_data(ev, nxt_cache_query_t, timer); - - cq->state->timeout_handler(thr, cq, NULL); -} - - -static void -nxt_cache_wake_handler(nxt_thread_t *thr, void *obj, void *data) -{ - nxt_cache_t *cache; - nxt_work_handler_t handler; - nxt_cache_query_t *q; - nxt_cache_query_wait_t *qw; - - qw = obj; - q = qw->query; - cache = qw->cache; - - nxt_cache_lock(cache); - - handler = nxt_cache_node_test(cache, q); - - if (handler != NULL) { - nxt_cache_query_wait_free(cache, qw); - - } else { - /* Wait again. */ - qw->next = q->node->waiting; - q->node->waiting = qw; - } - - nxt_cache_unlock(cache); - - handler(thr, q, NULL); -} - - -nxt_int_t -nxt_cache_update(nxt_cache_t *cache, nxt_cache_query_t *q) -{ - nxt_int_t ret; - nxt_cache_node_t *node; - nxt_lvlhsh_query_t lhq; - - node = q->node; - - node->accessed = nxt_cache_time(nxt_thread()) - cache->start_time; - - node->updating = 0; - node->count = 1; - - lhq.key_hash = nxt_murmur_hash2(node->key_data, node->key_len); - lhq.replace = 1; - lhq.key.len = node->key_len; - lhq.key.data = node->key_data; - lhq.value = node; - lhq.proto = cache->proto; - lhq.pool = cache->pool; - - nxt_cache_lock(cache); - - ret = nxt_lvlhsh_insert(&cache->lvlhsh, &lhq); - - if (nxt_fast_path(ret != NXT_OK)) { - - nxt_queue_insert_head(&cache->expiry_queue, &node->link); - - node = lhq.value; - - if (node != NULL) { - /* A replaced node. */ - - nxt_queue_remove(&node->link); - - if (node->count != 0) { - node->deleted = 1; - - } else { - // delete cache node - } - } - } - - nxt_cache_unlock(cache); - - return ret; -} - - -void -nxt_cache_release(nxt_cache_t *cache, nxt_cache_query_t *q) -{ - u_char *p, *data; - size_t size; - ssize_t ret; - nxt_thread_t *thr; - u_char buf[1024]; - - thr = nxt_thread(); - q->now = nxt_cache_time(thr); - - p = buf; - size = sizeof(buf); - - for ( ;; ) { - nxt_cache_lock(cache); - - ret = nxt_cache_release_locked(cache, q, p, size); - - nxt_cache_unlock(cache); - - if (ret == 0) { - return; - } - - size = nxt_abs(ret); - - data = nxt_malloc(size); - - if (data == NULL) { - /* TODO: retry */ - return; - } - - if (ret < 0) { - p = data; - continue; - } - - if (p != data) { - nxt_memcpy(data, p, size); - } - - nxt_thread_work_queue_add(thr, &thr->work_queue.main, - cache->delete_handler, data, NULL, thr->log); - } -} - - -static ssize_t -nxt_cache_release_locked(nxt_cache_t *cache, nxt_cache_query_t *q, - u_char *buf, size_t size) -{ - ssize_t ret; - nxt_cache_node_t *node; - - node = q->node; - node->count--; - - if (node->count != 0) { - return 0; - } - - if (!node->deleted) { - /* - * A cache node is locked whilst its count is non zero. - * To minimize number of operations the node's place in expiry - * queue can be updated only if the node is not currently used. - */ - node->accessed = q->now - cache->start_time; - - nxt_queue_remove(&node->link); - nxt_queue_insert_head(&cache->expiry_queue, &node->link); - - return 0; - } - - ret = 0; -#if 0 - - ret = cache->delete_copy(cache, node, buf, size); - - if (ret < 0) { - return ret; - } - -#endif - - nxt_cache_node_free(cache, node, 0); - - return ret; -} - - -static nxt_cache_node_t * -nxt_cache_node_alloc(nxt_cache_t *cache) -{ - nxt_queue_link_t *link; - nxt_cache_node_t *node; - - link = nxt_queue_first(&cache->free_nodes); - - if (nxt_fast_path(link != nxt_queue_tail(&cache->free_nodes))) { - cache->nfree_nodes--; - nxt_queue_remove(link); - - node = nxt_queue_link_data(link, nxt_cache_node_t, link); - nxt_memzero(node, sizeof(nxt_cache_node_t)); - - return node; - } - - nxt_cache_unlock(cache); - - node = cache->alloc(cache->data, sizeof(nxt_cache_node_t)); - - nxt_cache_lock(cache); - - return node; -} - - -static void -nxt_cache_node_free(nxt_cache_t *cache, nxt_cache_node_t *node, nxt_bool_t fast) -{ - if (fast || cache->nfree_nodes < 32) { - nxt_queue_insert_head(&cache->free_nodes, &node->link); - cache->nfree_nodes++; - return; - } - - nxt_cache_unlock(cache); - - cache->free(cache->data, node); - - nxt_cache_lock(cache); -} - - -static nxt_cache_query_wait_t * -nxt_cache_query_wait_alloc(nxt_cache_t *cache, nxt_bool_t *slow) -{ - nxt_cache_query_wait_t *qw; - - qw = cache->free_query_wait; - - if (nxt_fast_path(qw != NULL)) { - cache->free_query_wait = qw->next; - cache->nfree_query_wait--; - - *slow = 0; - return qw; - } - - nxt_cache_unlock(cache); - - qw = cache->alloc(cache->data, sizeof(nxt_cache_query_wait_t)); - *slow = 1; - - nxt_cache_lock(cache); - - return qw; -} - - -static void -nxt_cache_query_wait_free(nxt_cache_t *cache, nxt_cache_query_wait_t *qw) -{ - if (cache->nfree_query_wait < 32) { - qw->next = cache->free_query_wait; - cache->free_query_wait = qw; - cache->nfree_query_wait++; - return; - } - - nxt_cache_unlock(cache); - - cache->free(cache->data, qw); - - nxt_cache_lock(cache); -} diff --git a/src/nxt_cache.h b/src/nxt_cache.h deleted file mode 100644 index 567b5581..00000000 --- a/src/nxt_cache.h +++ /dev/null @@ -1,122 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_CACHE_INCLUDED_ -#define _NXT_CACHE_INCLUDED_ - - -typedef struct nxt_cache_query_s nxt_cache_query_t; -typedef struct nxt_cache_query_wait_s nxt_cache_query_wait_t; - - -typedef struct { - uint32_t shared; /* 1 bit */ - nxt_thread_spinlock_t lock; - - nxt_lvlhsh_t lvlhsh; - const nxt_lvlhsh_proto_t *proto; - void *pool; - - nxt_queue_t expiry_queue; - - nxt_queue_t free_nodes; - uint32_t nfree_nodes; - - uint32_t nfree_query_wait; - nxt_cache_query_wait_t *free_query_wait; - - uint64_t start_time; - - /* STUB: use nxt_lvlhsh_proto_t */ - void *(*alloc)(void *data, size_t size); - void (*free)(void *data, void *p); - void *data; - - nxt_work_handler_t delete_handler; -} nxt_cache_t; - - -typedef struct { - u_char *key_data; - - uint16_t key_len; /* 16 bits */ - uint8_t uses; /* 8 bits */ - uint8_t updating:1; - uint8_t deleted:1; - - uint32_t count; - - /* Times relative to the cache->start_time. */ - uint32_t expiry; - uint32_t accessed; - - nxt_off_t size; - - nxt_queue_link_t link; - - nxt_cache_query_wait_t *waiting; -} nxt_cache_node_t; - - -struct nxt_cache_query_wait_s { - nxt_cache_query_t *query; - nxt_cache_query_wait_t *next; - - uint8_t busy; /* 1 bit */ - uint8_t deleted; /* 1 bit */ - - nxt_pid_t pid; - nxt_event_engine_t *engine; - nxt_work_handler_t handler; - nxt_cache_t *cache; -}; - - -typedef struct { - nxt_work_handler_t nocache_handler; - nxt_work_handler_t ready_handler; - nxt_work_handler_t stale_handler; - nxt_work_handler_t update_stale_handler; - nxt_work_handler_t update_handler; - nxt_work_handler_t timeout_handler; - nxt_work_handler_t error_handler; -} nxt_cache_query_state_t; - - -struct nxt_cache_query_s { - u_char *key_data; - - uint16_t key_len; /* 16 bits */ -#if (NXT_64_BIT) - uint8_t hold; /* 1 bit */ - uint8_t use_stale; /* 1 bit */ - uint8_t update_stale; /* 1 bit */ - uint8_t stale; /* 1 bit */ -#else - uint8_t hold:1; - uint8_t use_stale:1; - uint8_t update_stale:1; - uint8_t stale:1; -#endif - - nxt_cache_node_t *node; - nxt_cache_query_t *next; - nxt_cache_query_state_t *state; - - nxt_time_t now; - - nxt_msec_t timeout; - nxt_timer_t timer; -}; - - -NXT_EXPORT void nxt_cache_init(nxt_cache_t *cache); -NXT_EXPORT void nxt_cache_query(nxt_cache_t *cache, nxt_cache_query_t *q); -NXT_EXPORT void nxt_cache_release(nxt_cache_t *cache, nxt_cache_query_t *q); -NXT_EXPORT nxt_int_t nxt_cache_update(nxt_cache_t *cache, nxt_cache_query_t *q); - - -#endif /* _NXT_CACHE_INCLUDED_ */ diff --git a/src/nxt_conf.c b/src/nxt_conf.c index d04aa45c..664b5468 100644 --- a/src/nxt_conf.c +++ b/src/nxt_conf.c @@ -46,7 +46,7 @@ typedef struct nxt_conf_object_s nxt_conf_object_t; struct nxt_conf_value_s { union { uint8_t boolean; /* 1 bit. */ - u_char number[NXT_CONF_MAX_NUMBER_LEN + 1];; + u_char number[NXT_CONF_MAX_NUMBER_LEN + 1]; struct { u_char start[NXT_CONF_MAX_SHORT_STRING]; diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 537a3fb7..8c75a9fe 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -7,6 +7,7 @@ #include <nxt_main.h> #include <nxt_conf.h> #include <nxt_cert.h> +#include <nxt_script.h> #include <nxt_router.h> #include <nxt_http.h> #include <nxt_sockaddr.h> @@ -226,6 +227,13 @@ static nxt_int_t nxt_conf_vldt_cgroup_path(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); #endif +#if (NXT_HAVE_NJS) +static nxt_int_t nxt_conf_vldt_js_module(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value, void *data); +static nxt_int_t nxt_conf_vldt_js_module_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value); +#endif + static nxt_conf_vldt_object_t nxt_conf_vldt_setting_members[]; static nxt_conf_vldt_object_t nxt_conf_vldt_http_members[]; @@ -297,6 +305,12 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_setting_members[] = { .type = NXT_CONF_VLDT_OBJECT, .validator = nxt_conf_vldt_object, .u.members = nxt_conf_vldt_http_members, +#if (NXT_HAVE_NJS) + }, { + .name = nxt_string("js_module"), + .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, + .validator = nxt_conf_vldt_js_module, +#endif }, NXT_CONF_VLDT_END @@ -344,6 +358,12 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_http_members[] = { .type = NXT_CONF_VLDT_OBJECT, .validator = nxt_conf_vldt_object, .u.members = nxt_conf_vldt_static_members, + }, { + .name = nxt_string("log_route"), + .type = NXT_CONF_VLDT_BOOLEAN, + }, { + .name = nxt_string("server_version"), + .type = NXT_CONF_VLDT_BOOLEAN, }, NXT_CONF_VLDT_END @@ -663,6 +683,16 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_match_members[] = { }; +static nxt_conf_vldt_object_t nxt_conf_vldt_action_common_members[] = { + { + .name = nxt_string("rewrite"), + .type = NXT_CONF_VLDT_STRING, + }, + + NXT_CONF_VLDT_END +}; + + static nxt_conf_vldt_object_t nxt_conf_vldt_pass_action_members[] = { { .name = nxt_string("pass"), @@ -671,7 +701,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_pass_action_members[] = { .flags = NXT_CONF_VLDT_TSTR, }, - NXT_CONF_VLDT_END + NXT_CONF_VLDT_NEXT(nxt_conf_vldt_action_common_members) }; @@ -686,7 +716,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_return_action_members[] = { .flags = NXT_CONF_VLDT_TSTR, }, - NXT_CONF_VLDT_END + NXT_CONF_VLDT_NEXT(nxt_conf_vldt_action_common_members) }; @@ -730,7 +760,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { #endif }, - NXT_CONF_VLDT_END + NXT_CONF_VLDT_NEXT(nxt_conf_vldt_action_common_members) }; @@ -741,7 +771,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_proxy_action_members[] = { .validator = nxt_conf_vldt_proxy, }, - NXT_CONF_VLDT_END + NXT_CONF_VLDT_NEXT(nxt_conf_vldt_action_common_members) }; @@ -1044,6 +1074,12 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_common_members[] = { .type = NXT_CONF_VLDT_OBJECT, .validator = nxt_conf_vldt_isolation, .u.members = nxt_conf_vldt_app_isolation_members, + }, { + .name = nxt_string("stdout"), + .type = NXT_CONF_VLDT_STRING, + }, { + .name = nxt_string("stderr"), + .type = NXT_CONF_VLDT_STRING, }, NXT_CONF_VLDT_END @@ -1284,35 +1320,26 @@ nxt_conf_validate(nxt_conf_validation_t *vldt) vldt->tstr_state = nxt_tstr_state_new(vldt->pool, 1); if (nxt_slow_path(vldt->tstr_state == NULL)) { - ret = NXT_ERROR; - goto fail; + return NXT_ERROR; } ret = nxt_conf_vldt_type(vldt, NULL, vldt->conf, NXT_CONF_VLDT_OBJECT); if (ret != NXT_OK) { - goto fail; + return ret; } ret = nxt_conf_vldt_object(vldt, vldt->conf, nxt_conf_vldt_root_members); if (ret != NXT_OK) { - goto fail; + return ret; } ret = nxt_tstr_state_done(vldt->tstr_state, error); if (ret != NXT_OK) { ret = nxt_conf_vldt_error(vldt, "%s", error); - goto fail; + return ret; } - nxt_tstr_state_release(vldt->tstr_state); - return NXT_OK; - -fail: - - nxt_tstr_state_release(vldt->tstr_state); - - return ret; } @@ -3219,6 +3246,49 @@ nxt_conf_vldt_server_weight(nxt_conf_validation_t *vldt, } +#if (NXT_HAVE_NJS) + +static nxt_int_t +nxt_conf_vldt_js_module(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, + void *data) +{ + if (nxt_conf_type(value) == NXT_CONF_ARRAY) { + return nxt_conf_vldt_array_iterator(vldt, value, + &nxt_conf_vldt_js_module_element); + } + + /* NXT_CONF_STRING */ + + return nxt_conf_vldt_js_module_element(vldt, value); +} + + +static nxt_int_t +nxt_conf_vldt_js_module_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value) +{ + nxt_str_t name; + nxt_conf_value_t *module; + + if (nxt_conf_type(value) != NXT_CONF_STRING) { + return nxt_conf_vldt_error(vldt, "The \"js_module\" array must " + "contain only string values."); + } + + nxt_conf_get_string(value, &name); + + module = nxt_script_info_get(&name); + if (module == NULL) { + return nxt_conf_vldt_error(vldt, "JS module \"%V\" is not found.", + &name); + } + + return NXT_OK; +} + +#endif + + typedef struct { nxt_str_t path; nxt_str_t format; diff --git a/src/nxt_controller.c b/src/nxt_controller.c index b5e0d831..4e2e3749 100644 --- a/src/nxt_controller.c +++ b/src/nxt_controller.c @@ -11,6 +11,7 @@ #include <nxt_conf.h> #include <nxt_status.h> #include <nxt_cert.h> +#include <nxt_script.h> typedef struct { @@ -101,6 +102,15 @@ static nxt_bool_t nxt_controller_cert_in_use(nxt_str_t *name); static void nxt_controller_cert_cleanup(nxt_task_t *task, void *obj, void *data); #endif +#if (NXT_HAVE_NJS) +static void nxt_controller_process_script(nxt_task_t *task, + nxt_controller_request_t *req, nxt_str_t *path); +static void nxt_controller_process_script_save(nxt_task_t *task, + nxt_port_recv_msg_t *msg, void *data); +static nxt_bool_t nxt_controller_script_in_use(nxt_str_t *name); +static void nxt_controller_script_cleanup(nxt_task_t *task, void *obj, + void *data); +#endif static void nxt_controller_process_control(nxt_task_t *task, nxt_controller_request_t *req, nxt_str_t *path); static void nxt_controller_app_restart_handler(nxt_task_t *task, @@ -213,6 +223,13 @@ nxt_controller_prefork(nxt_task_t *task, nxt_process_t *process, nxt_mp_t *mp) nxt_mp_cleanup(mp, nxt_controller_cert_cleanup, task, ctrl_init.certs, rt); #endif +#if (NXT_HAVE_NJS) + ctrl_init.scripts = nxt_script_store_load(task, mp); + + nxt_mp_cleanup(mp, nxt_controller_script_cleanup, task, ctrl_init.scripts, + rt); +#endif + process->data.controller = ctrl_init; return NXT_OK; @@ -321,6 +338,13 @@ nxt_controller_start(nxt_task_t *task, nxt_process_data_t *data) } #endif +#if (NXT_HAVE_NJS) + if (init->scripts != NULL) { + nxt_script_info_init(task, init->scripts); + nxt_script_store_release(init->scripts); + } +#endif + json = &init->conf; if (json->start == NULL) { @@ -1047,9 +1071,19 @@ nxt_controller_process_request(nxt_task_t *task, nxt_controller_request_t *req) nxt_controller_response_t resp; #if (NXT_TLS) nxt_conf_value_t *certs; +#endif +#if (NXT_HAVE_NJS) + nxt_conf_value_t *scripts; +#endif +#if (NXT_TLS) static nxt_str_t certificates = nxt_string("certificates"); #endif + +#if (NXT_HAVE_NJS) + static nxt_str_t scripts_str = nxt_string("js_modules"); +#endif + static nxt_str_t config = nxt_string("config"); static nxt_str_t status = nxt_string("status"); @@ -1120,6 +1154,25 @@ nxt_controller_process_request(nxt_task_t *task, nxt_controller_request_t *req) #endif +#if (NXT_HAVE_NJS) + + if (nxt_str_start(&path, "/js_modules", 11) + && (path.length == 11 || path.start[11] == '/')) + { + if (path.length == 11) { + path.length = 1; + + } else { + path.length -= 11; + path.start += 11; + } + + nxt_controller_process_script(task, req, &path); + return; + } + +#endif + if (nxt_str_start(&path, "/control/", 9)) { path.length -= 9; path.start += 9; @@ -1143,6 +1196,9 @@ nxt_controller_process_request(nxt_task_t *task, nxt_controller_request_t *req) #if (NXT_TLS) count++; #endif +#if (NXT_HAVE_NJS) + count++; +#endif value = nxt_conf_create_object(c->mem_pool, count); if (nxt_slow_path(value == NULL)) { @@ -1160,6 +1216,15 @@ nxt_controller_process_request(nxt_task_t *task, nxt_controller_request_t *req) nxt_conf_set_member(value, &certificates, certs, i++); #endif +#if (NXT_HAVE_NJS) + scripts = nxt_script_info_get_all(c->mem_pool); + if (nxt_slow_path(scripts == NULL)) { + goto alloc_fail; + } + + nxt_conf_set_member(value, &scripts_str, scripts, i++); +#endif + nxt_conf_set_member(value, &config, nxt_controller_conf.root, i++); nxt_conf_set_member(value, &status, nxt_controller_status, i); @@ -1879,6 +1944,294 @@ nxt_controller_cert_in_use(nxt_str_t *name) #endif +#if (NXT_HAVE_NJS) + +static void +nxt_controller_process_script(nxt_task_t *task, + nxt_controller_request_t *req, nxt_str_t *path) +{ + u_char *p; + nxt_int_t ret; + nxt_str_t name; + nxt_conn_t *c; + nxt_script_t *script; + nxt_buf_mem_t *bm; + nxt_conf_value_t *value; + nxt_controller_response_t resp; + u_char error[NXT_MAX_ERROR_STR]; + + name.length = path->length - 1; + name.start = path->start + 1; + + p = memchr(name.start, '/', name.length); + + if (p != NULL) { + name.length = p - name.start; + + path->length -= p - path->start; + path->start = p; + + } else { + path = NULL; + } + + nxt_memzero(&resp, sizeof(nxt_controller_response_t)); + + c = req->conn; + + if (nxt_str_eq(&req->parser.method, "GET", 3)) { + + if (name.length != 0) { + value = nxt_script_info_get(&name); + if (value == NULL) { + goto script_not_found; + } + + if (path != NULL) { + value = nxt_conf_get_path(value, path); + if (value == NULL) { + goto not_found; + } + } + + } else { + value = nxt_script_info_get_all(c->mem_pool); + if (value == NULL) { + goto alloc_fail; + } + } + + resp.status = 200; + resp.conf = value; + + nxt_controller_response(task, req, &resp); + return; + } + + if (name.length == 0 || path != NULL) { + goto invalid_name; + } + + if (nxt_str_eq(&req->parser.method, "PUT", 3)) { + value = nxt_script_info_get(&name); + if (value != NULL) { + goto exists_script; + } + + bm = &c->read->mem; + + script = nxt_script_new(task, &name, bm->pos, + nxt_buf_mem_used_size(bm), error); + if (script == NULL) { + goto invalid_script; + } + + ret = nxt_script_info_save(&name, script); + + nxt_script_destroy(script); + + if (nxt_slow_path(ret != NXT_OK)) { + goto alloc_fail; + } + + nxt_script_store_get(task, &name, c->mem_pool, + nxt_controller_process_script_save, req); + return; + } + + if (nxt_str_eq(&req->parser.method, "DELETE", 6)) { + + if (nxt_controller_script_in_use(&name)) { + goto script_in_use; + } + + if (nxt_script_info_delete(&name) != NXT_OK) { + goto script_not_found; + } + + nxt_script_store_delete(task, &name, c->mem_pool); + + resp.status = 200; + resp.title = (u_char *) "JS module deleted."; + + nxt_controller_response(task, req, &resp); + return; + } + + resp.status = 405; + resp.title = (u_char *) "Invalid method."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +invalid_name: + + resp.status = 400; + resp.title = (u_char *) "Invalid JS module name."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +invalid_script: + + resp.status = 400; + resp.title = (u_char *) "Invalid JS module."; + resp.offset = -1; + + resp.detail.start = error; + resp.detail.length = nxt_strlen(error); + + nxt_controller_response(task, req, &resp); + return; + +exists_script: + + resp.status = 400; + resp.title = (u_char *) "JS module already exists."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +script_in_use: + + resp.status = 400; + resp.title = (u_char *) "JS module is used in the configuration."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +script_not_found: + + resp.status = 404; + resp.title = (u_char *) "JS module doesn't exist."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +not_found: + + resp.status = 404; + resp.title = (u_char *) "Invalid path."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); + return; + +alloc_fail: + + resp.status = 500; + resp.title = (u_char *) "Memory allocation failed."; + resp.offset = -1; + + nxt_controller_response(task, req, &resp); +} + + +static void +nxt_controller_process_script_save(nxt_task_t *task, nxt_port_recv_msg_t *msg, + void *data) +{ + nxt_conn_t *c; + nxt_buf_mem_t *mbuf; + nxt_controller_request_t *req; + nxt_controller_response_t resp; + + req = data; + + nxt_memzero(&resp, sizeof(nxt_controller_response_t)); + + if (msg == NULL || msg->port_msg.type == _NXT_PORT_MSG_RPC_ERROR) { + resp.status = 500; + resp.title = (u_char *) "Failed to store script."; + + nxt_controller_response(task, req, &resp); + return; + } + + c = req->conn; + + mbuf = &c->read->mem; + + nxt_fd_write(msg->fd[0], mbuf->pos, nxt_buf_mem_used_size(mbuf)); + + nxt_fd_close(msg->fd[0]); + + nxt_memzero(&resp, sizeof(nxt_controller_response_t)); + + resp.status = 200; + resp.title = (u_char *) "JS module uploaded."; + + nxt_controller_response(task, req, &resp); +} + + +static nxt_bool_t +nxt_controller_script_in_use(nxt_str_t *name) +{ + uint32_t i, n; + nxt_str_t str; + nxt_conf_value_t *js_module, *element; + + static nxt_str_t js_module_path = nxt_string("/settings/js_module"); + + js_module = nxt_conf_get_path(nxt_controller_conf.root, + &js_module_path); + + if (js_module != NULL) { + + if (nxt_conf_type(js_module) == NXT_CONF_ARRAY) { + n = nxt_conf_array_elements_count(js_module); + + for (i = 0; i < n; i++) { + element = nxt_conf_get_array_element(js_module, i); + + nxt_conf_get_string(element, &str); + + if (nxt_strstr_eq(&str, name)) { + return 1; + } + } + + } else { + /* NXT_CONF_STRING */ + + nxt_conf_get_string(js_module, &str); + + if (nxt_strstr_eq(&str, name)) { + return 1; + } + } + } + + return 0; +} + + +static void +nxt_controller_script_cleanup(nxt_task_t *task, void *obj, void *data) +{ + pid_t main_pid; + nxt_array_t *scripts; + nxt_runtime_t *rt; + + scripts = obj; + rt = data; + + main_pid = rt->port_by_type[NXT_PROCESS_MAIN]->pid; + + if (nxt_pid == main_pid && scripts != NULL) { + nxt_script_store_release(scripts); + } +} + +#endif + + static void nxt_controller_conf_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg, void *data) diff --git a/src/nxt_fastcgi_record_parse.c b/src/nxt_fastcgi_record_parse.c deleted file mode 100644 index 7d2ce32e..00000000 --- a/src/nxt_fastcgi_record_parse.c +++ /dev/null @@ -1,307 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -#define NXT_FASTCGI_DATA_MIDDLE 0 -#define NXT_FASTCGI_DATA_END_ON_BORDER 1 -#define NXT_FASTCGI_DATA_END 2 - - -static nxt_int_t nxt_fastcgi_buffer(nxt_fastcgi_parse_t *fp, nxt_buf_t ***tail, - nxt_buf_t *in); - - -void -nxt_fastcgi_record_parse(nxt_task_t *task, nxt_fastcgi_parse_t *fp, - nxt_buf_t *in) -{ - u_char ch; - nxt_int_t ret, stream; - nxt_buf_t *b, *nb, **tail[2]; - const char *msg; - enum { - sw_fastcgi_version = 0, - sw_fastcgi_type, - sw_fastcgi_request_id_high, - sw_fastcgi_request_id_low, - sw_fastcgi_content_length_high, - sw_fastcgi_content_length_low, - sw_fastcgi_padding_length, - sw_fastcgi_reserved, - sw_fastcgi_data, - sw_fastcgi_padding, - sw_fastcgi_end_request, - } state; - - fp->out[0] = NULL; - fp->out[1] = NULL; - - tail[0] = &fp->out[0]; - tail[1] = &fp->out[1]; - - state = fp->state; - - for (b = in; b != NULL; b = b->next) { - - if (nxt_buf_is_sync(b)) { - **tail = b; - *tail = &b->next; - continue; - } - - fp->pos = b->mem.pos; - - while (fp->pos < b->mem.free) { - /* - * The sw_fastcgi_data state is tested outside the - * switch to preserve fp->pos and to not touch memory. - */ - if (state == sw_fastcgi_data) { - - /* - * fp->type here can be only NXT_FASTCGI_STDOUT - * or NXT_FASTCGI_STDERR. NXT_FASTCGI_END_REQUEST - * is tested in sw_fastcgi_reserved. - */ - stream = fp->type - NXT_FASTCGI_STDOUT; - - ret = nxt_fastcgi_buffer(fp, &tail[stream], b); - - if (ret == NXT_FASTCGI_DATA_MIDDLE) { - goto next; - } - - if (nxt_slow_path(ret == NXT_ERROR)) { - fp->error = 1; - goto done; - } - - if (fp->padding == 0) { - state = sw_fastcgi_version; - - } else { - state = sw_fastcgi_padding; - } - - if (ret == NXT_FASTCGI_DATA_END_ON_BORDER) { - goto next; - } - - /* ret == NXT_FASTCGI_DATA_END */ - } - - ch = *fp->pos++; - - nxt_thread_log_debug("fastcgi record byte: %02Xd", ch); - - switch (state) { - - case sw_fastcgi_version: - if (nxt_fast_path(ch == 1)) { - state = sw_fastcgi_type; - continue; - } - - msg = "unsupported FastCGI protocol version"; - goto fastcgi_error; - - case sw_fastcgi_type: - switch (ch) { - case NXT_FASTCGI_STDOUT: - case NXT_FASTCGI_STDERR: - case NXT_FASTCGI_END_REQUEST: - fp->type = ch; - state = sw_fastcgi_request_id_high; - continue; - default: - msg = "invalid FastCGI record type"; - goto fastcgi_error; - } - - case sw_fastcgi_request_id_high: - /* FastCGI multiplexing is not supported. */ - if (nxt_fast_path(ch == 0)) { - state = sw_fastcgi_request_id_low; - continue; - } - - msg = "unexpected FastCGI request ID high byte"; - goto fastcgi_error; - - case sw_fastcgi_request_id_low: - if (nxt_fast_path(ch == 1)) { - state = sw_fastcgi_content_length_high; - continue; - } - - msg = "unexpected FastCGI request ID low byte"; - goto fastcgi_error; - - case sw_fastcgi_content_length_high: - fp->length = ch << 8; - state = sw_fastcgi_content_length_low; - continue; - - case sw_fastcgi_content_length_low: - fp->length |= ch; - state = sw_fastcgi_padding_length; - continue; - - case sw_fastcgi_padding_length: - fp->padding = ch; - state = sw_fastcgi_reserved; - continue; - - case sw_fastcgi_reserved: - nxt_thread_log_debug("fastcgi record type:%d " - "length:%uz padding:%d", - fp->type, fp->length, fp->padding); - - if (nxt_fast_path(fp->type != NXT_FASTCGI_END_REQUEST)) { - state = sw_fastcgi_data; - continue; - } - - state = sw_fastcgi_end_request; - continue; - - case sw_fastcgi_data: - /* - * This state is processed before the switch. - * It added here just to suppress a warning. - */ - continue; - - case sw_fastcgi_padding: - /* - * No special fast processing of padding - * because it usually takes just 1-7 bytes. - */ - fp->padding--; - - if (fp->padding == 0) { - nxt_thread_log_debug("fastcgi record end"); - state = sw_fastcgi_version; - } - continue; - - case sw_fastcgi_end_request: - /* Just skip 8 bytes of END_REQUEST. */ - fp->length--; - - if (fp->length != 0) { - continue; - } - - fp->done = 1; - - nxt_thread_log_debug("fastcgi end request"); - - goto done; - } - } - - if (b->retain == 0) { - /* No record data was found in a buffer. */ - nxt_thread_current_work_queue_add(task->thread, - b->completion_handler, - task, b, b->parent); - } - - next: - - continue; - } - - fp->state = state; - - return; - -fastcgi_error: - - nxt_thread_log_error(NXT_LOG_ERR, "upstream sent %s: %d", msg, ch); - - fp->fastcgi_error = 1; - -done: - - nb = fp->last_buf(fp); - - if (nxt_fast_path(nb != NULL)) { - *tail[0] = nb; - - } else { - fp->error = 1; - } - - // STUB: fp->fastcgi_error = 1; - // STUB: fp->error = 1; - - return; -} - - -static nxt_int_t -nxt_fastcgi_buffer(nxt_fastcgi_parse_t *fp, nxt_buf_t ***tail, nxt_buf_t *in) -{ - u_char *p; - size_t size; - nxt_buf_t *b; - - if (fp->length == 0) { - return NXT_FASTCGI_DATA_END; - } - - p = fp->pos; - size = in->mem.free - p; - - if (fp->length >= size && in->retain == 0) { - /* - * Use original buffer if the buffer is lesser than or equal to - * FastCGI record size and this is the first record in the buffer. - */ - in->mem.pos = p; - **tail = in; - *tail = &in->next; - - } else { - b = nxt_buf_mem_alloc(fp->mem_pool, 0, 0); - if (nxt_slow_path(b == NULL)) { - return NXT_ERROR; - } - - **tail = b; - *tail = &b->next; - - b->parent = in; - in->retain++; - b->mem.pos = p; - b->mem.start = p; - - if (fp->length < size) { - p += fp->length; - fp->pos = p; - - b->mem.free = p; - b->mem.end = p; - - return NXT_FASTCGI_DATA_END; - } - - b->mem.free = in->mem.free; - b->mem.end = in->mem.free; - } - - fp->length -= size; - - if (fp->length == 0) { - return NXT_FASTCGI_DATA_END_ON_BORDER; - } - - return NXT_FASTCGI_DATA_MIDDLE; -} diff --git a/src/nxt_fastcgi_source.c b/src/nxt_fastcgi_source.c deleted file mode 100644 index b2424292..00000000 --- a/src/nxt_fastcgi_source.c +++ /dev/null @@ -1,750 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -#define NXT_FASTCGI_RESPONDER 1 -#define NXT_FASTCGI_KEEP_CONN 1 - - -typedef struct { - u_char *buf; - uint32_t len; - u_char length[4]; -} nxt_fastcgi_param_t; - - -#define nxt_fastcgi_set_record_length(p, length) \ - do { \ - uint32_t len = length; \ - \ - p[1] = (u_char) len; len >>= 8; \ - p[0] = (u_char) len; \ - } while (0) - - -nxt_inline size_t -nxt_fastcgi_param_length(u_char *p, uint32_t length) -{ - if (nxt_fast_path(length < 128)) { - *p = (u_char) length; - return 1; - } - - p[3] = (u_char) length; length >>= 8; - p[2] = (u_char) length; length >>= 8; - p[1] = (u_char) length; length >>= 8; - p[0] = (u_char) (length | 0x80); - - return 4; -} - - -static nxt_buf_t *nxt_fastcgi_request_create(nxt_fastcgi_source_t *fs); -static nxt_int_t nxt_fastcgi_next_param(nxt_fastcgi_source_t *fs, - nxt_fastcgi_param_t *param); - -static void nxt_fastcgi_source_record_filter(nxt_task_t *task, void *obj, - void *data); -static void nxt_fastcgi_source_record_error(nxt_task_t *task, void *obj, - void *data); -static void nxt_fastcgi_source_header_filter(nxt_task_t *task, void *obj, - void *data); -static void nxt_fastcgi_source_sync_buffer(nxt_task_t *task, - nxt_fastcgi_source_t *fs, nxt_buf_t *b); - -static nxt_int_t nxt_fastcgi_source_header_process(nxt_task_t *task, - nxt_fastcgi_source_t *fs); -static nxt_int_t nxt_fastcgi_source_status(nxt_upstream_source_t *us, - nxt_name_value_t *nv); -static nxt_int_t nxt_fastcgi_source_content_length(nxt_upstream_source_t *us, - nxt_name_value_t *nv); - -static void nxt_fastcgi_source_header_ready(nxt_fastcgi_source_t *fs, - nxt_buf_t *b); -static void nxt_fastcgi_source_body_filter(nxt_task_t *task, void *obj, - void *data); -static nxt_buf_t *nxt_fastcgi_source_last_buf(nxt_fastcgi_parse_t *fp); -static void nxt_fastcgi_source_error(nxt_task_t *task, - nxt_stream_source_t *stream); -static void nxt_fastcgi_source_fail(nxt_task_t *task, nxt_fastcgi_source_t *fs); - - -/* - * A FastCGI request: - * FCGI_BEGIN_REQUEST record; - * Several FCGI_PARAMS records, the last FCGI_PARAMS record must have - * zero content length, - * Several FCGI_STDIN records, the last FCGI_STDIN record must have - * zero content length. - */ - -static const uint8_t nxt_fastcgi_begin_request[] = { - 1, /* FastCGI version. */ - NXT_FASTCGI_BEGIN_REQUEST, /* The BEGIN_REQUEST record type. */ - 0, 1, /* Request ID. */ - 0, 8, /* Content length of the Role record. */ - 0, /* Padding length. */ - 0, /* Reserved. */ - - 0, NXT_FASTCGI_RESPONDER, /* The Responder Role. */ - 0, /* Flags. */ - 0, 0, 0, 0, 0, /* Reserved. */ -}; - - -static const uint8_t nxt_fastcgi_params_record[] = { - 1, /* FastCGI version. */ - NXT_FASTCGI_PARAMS, /* The PARAMS record type. */ - 0, 1, /* Request ID. */ - 0, 0, /* Content length. */ - 0, /* Padding length. */ - 0, /* Reserved. */ -}; - - -static const uint8_t nxt_fastcgi_stdin_record[] = { - 1, /* FastCGI version. */ - NXT_FASTCGI_STDIN, /* The STDIN record type. */ - 0, 1, /* Request ID. */ - 0, 0, /* Content length. */ - 0, /* Padding length. */ - 0, /* Reserved. */ -}; - - -void -nxt_fastcgi_source_handler(nxt_task_t *task, nxt_upstream_source_t *us, - nxt_fastcgi_source_request_create_t request_create) -{ - nxt_stream_source_t *stream; - nxt_fastcgi_source_t *fs; - - fs = nxt_mp_zget(us->buffers.mem_pool, sizeof(nxt_fastcgi_source_t)); - if (nxt_slow_path(fs == NULL)) { - goto fail; - } - - us->protocol_source = fs; - - fs->header_in.list = nxt_list_create(us->buffers.mem_pool, 8, - sizeof(nxt_name_value_t)); - if (nxt_slow_path(fs->header_in.list == NULL)) { - goto fail; - } - - fs->header_in.hash = us->header_hash; - fs->upstream = us; - fs->request_create = request_create; - - stream = us->stream; - - if (stream == NULL) { - stream = nxt_mp_zget(us->buffers.mem_pool, sizeof(nxt_stream_source_t)); - if (nxt_slow_path(stream == NULL)) { - goto fail; - } - - us->stream = stream; - stream->upstream = us; - - } else { - nxt_memzero(stream, sizeof(nxt_stream_source_t)); - } - - /* - * Create the FastCGI source filter chain: - * stream source | FastCGI record filter | FastCGI HTTP header filter - */ - stream->next = &fs->query; - stream->error_handler = nxt_fastcgi_source_error; - - fs->record.next.context = fs; - fs->record.next.filter = nxt_fastcgi_source_header_filter; - - fs->record.parse.last_buf = nxt_fastcgi_source_last_buf; - fs->record.parse.data = fs; - fs->record.parse.mem_pool = us->buffers.mem_pool; - - fs->query.context = &fs->record.parse; - fs->query.filter = nxt_fastcgi_source_record_filter; - - fs->header_in.content_length = -1; - - stream->out = nxt_fastcgi_request_create(fs); - - if (nxt_fast_path(stream->out != NULL)) { - nxt_memzero(&fs->u.header, sizeof(nxt_http_split_header_parse_t)); - fs->u.header.mem_pool = fs->upstream->buffers.mem_pool; - - nxt_stream_source_connect(task, stream); - return; - } - -fail: - - nxt_fastcgi_source_fail(task, fs); -} - - -static nxt_buf_t * -nxt_fastcgi_request_create(nxt_fastcgi_source_t *fs) -{ - u_char *p, *record_length; - size_t len, size, max_record_size; - nxt_int_t ret; - nxt_buf_t *b, *req, **prev; - nxt_bool_t begin_request; - nxt_fastcgi_param_t param; - - nxt_thread_log_debug("fastcgi request"); - - begin_request = 1; - param.len = 0; - prev = &req; - -new_buffer: - - ret = nxt_buf_pool_mem_alloc(&fs->upstream->buffers, 0); - if (nxt_slow_path(ret != NXT_OK)) { - return NULL; - } - - b = fs->upstream->buffers.current; - fs->upstream->buffers.current = NULL; - - *prev = b; - prev = &b->next; - -new_record: - - size = b->mem.end - b->mem.free; - size = nxt_align_size(size, 8) - 8; - /* The maximal FastCGI record content size is 65535. 65528 is 64K - 8. */ - max_record_size = nxt_min(65528, size); - - p = b->mem.free; - - if (begin_request) { - /* TODO: fastcgi keep conn in flags. */ - p = nxt_cpymem(p, nxt_fastcgi_begin_request, 16); - max_record_size -= 16; - begin_request = 0; - } - - b->mem.free = nxt_cpymem(p, nxt_fastcgi_params_record, 8); - record_length = &p[4]; - size = 0; - - for ( ;; ) { - if (param.len == 0) { - ret = nxt_fastcgi_next_param(fs, ¶m); - - if (nxt_slow_path(ret != NXT_OK)) { - - if (nxt_slow_path(ret == NXT_ERROR)) { - return NULL; - } - - /* ret == NXT_DONE */ - break; - } - } - - len = max_record_size; - - if (nxt_fast_path(len >= param.len)) { - len = param.len; - param.len = 0; - - } else { - param.len -= len; - } - - nxt_thread_log_debug("fastcgi copy len:%uz", len); - - b->mem.free = nxt_cpymem(b->mem.free, param.buf, len); - - size += len; - max_record_size -= len; - - if (nxt_slow_path(param.len != 0)) { - /* The record is full. */ - - param.buf += len; - - nxt_thread_log_debug("fastcgi content size:%uz", size); - - nxt_fastcgi_set_record_length(record_length, size); - - /* The minimal size of aligned record with content is 16 bytes. */ - if (b->mem.end - b->mem.free >= 16) { - goto new_record; - } - - nxt_thread_log_debug("\"%*s\"", b->mem.free - b->mem.pos, - b->mem.pos); - goto new_buffer; - } - } - - nxt_thread_log_debug("fastcgi content size:%uz", size); - - nxt_fastcgi_set_record_length(record_length, size); - - /* A padding length. */ - size = 8 - size % 8; - record_length[2] = (u_char) size; - nxt_memzero(b->mem.free, size); - b->mem.free += size; - - nxt_thread_log_debug("fastcgi padding:%uz", size); - - if (b->mem.end - b->mem.free < 16) { - nxt_thread_log_debug("\"%*s\"", b->mem.free - b->mem.pos, b->mem.pos); - - b = nxt_buf_mem_alloc(fs->upstream->buffers.mem_pool, 16, 0); - if (nxt_slow_path(b == NULL)) { - return NULL; - } - - *prev = b; - prev = &b->next; - } - - /* The end of FastCGI params. */ - p = nxt_cpymem(b->mem.free, nxt_fastcgi_params_record, 8); - - /* The end of FastCGI stdin. */ - b->mem.free = nxt_cpymem(p, nxt_fastcgi_stdin_record, 8); - - nxt_thread_log_debug("\"%*s\"", b->mem.free - b->mem.pos, b->mem.pos); - - return req; -} - - -static nxt_int_t -nxt_fastcgi_next_param(nxt_fastcgi_source_t *fs, nxt_fastcgi_param_t *param) -{ - nxt_int_t ret; - - enum { - sw_name_length = 0, - sw_value_length, - sw_name, - sw_value, - }; - - switch (fs->state) { - - case sw_name_length: - ret = fs->request_create(fs); - - if (nxt_slow_path(ret != NXT_OK)) { - return ret; - } - - nxt_thread_log_debug("fastcgi param \"%V: %V\"", - &fs->u.request.name, &fs->u.request.value); - - fs->state = sw_value_length; - param->buf = param->length; - param->len = nxt_fastcgi_param_length(param->length, - fs->u.request.name.len); - break; - - case sw_value_length: - fs->state = sw_name; - param->buf = param->length; - param->len = nxt_fastcgi_param_length(param->length, - fs->u.request.value.len); - break; - - case sw_name: - fs->state = sw_value; - param->buf = fs->u.request.name.data; - param->len = fs->u.request.name.len; - break; - - case sw_value: - fs->state = sw_name_length; - param->buf = fs->u.request.value.data; - param->len = fs->u.request.value.len; - break; - } - - return NXT_OK; -} - - -static void -nxt_fastcgi_source_record_filter(nxt_task_t *task, void *obj, void *data) -{ - size_t size; - u_char *p; - nxt_buf_t *b, *in; - nxt_fastcgi_source_t *fs; - nxt_fastcgi_source_record_t *fsr; - - fsr = obj; - in = data; - - nxt_debug(task, "fastcgi source record filter"); - - if (nxt_slow_path(fsr->parse.done)) { - return; - } - - nxt_fastcgi_record_parse(task, &fsr->parse, in); - - fs = nxt_container_of(fsr, nxt_fastcgi_source_t, record); - - if (fsr->parse.error) { - nxt_fastcgi_source_fail(task, fs); - return; - } - - if (fsr->parse.fastcgi_error) { - /* - * Output all parsed before a FastCGI record error and close upstream. - */ - nxt_thread_current_work_queue_add(task->thread, - nxt_fastcgi_source_record_error, - task, fs, NULL); - } - - /* Log FastCGI stderr output. */ - - for (b = fsr->parse.out[1]; b != NULL; b = b->next) { - - for (p = b->mem.free - 1; p >= b->mem.pos; p--) { - if (*p != '\r' && *p != '\n') { - break; - } - } - - size = (p + 1) - b->mem.pos; - - if (size != 0) { - nxt_log(task, NXT_LOG_ERR, - "upstream sent in FastCGI stderr: \"%*s\"", - size, b->mem.pos); - } - - b->completion_handler(task, b, b->parent); - } - - /* Process FastCGI stdout output. */ - - if (fsr->parse.out[0] != NULL) { - nxt_source_filter(task->thread, fs->upstream->work_queue, task, - &fsr->next, fsr->parse.out[0]); - } -} - - -static void -nxt_fastcgi_source_record_error(nxt_task_t *task, void *obj, void *data) -{ - nxt_fastcgi_source_t *fs; - - fs = obj; - - nxt_fastcgi_source_fail(task, fs); -} - - -static void -nxt_fastcgi_source_header_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_int_t ret; - nxt_buf_t *b; - nxt_fastcgi_source_t *fs; - - fs = obj; - b = data; - - do { - nxt_debug(task, "fastcgi source header filter"); - - if (nxt_slow_path(nxt_buf_is_sync(b))) { - nxt_fastcgi_source_sync_buffer(task, fs, b); - return; - } - - for ( ;; ) { - ret = nxt_http_split_header_parse(&fs->u.header, &b->mem); - - if (nxt_slow_path(ret != NXT_OK)) { - break; - } - - ret = nxt_fastcgi_source_header_process(task, fs); - - if (nxt_slow_path(ret != NXT_OK)) { - break; - } - } - - if (nxt_fast_path(ret == NXT_DONE)) { - nxt_debug(task, "fastcgi source header done"); - nxt_fastcgi_source_header_ready(fs, b); - return; - } - - if (nxt_fast_path(ret != NXT_AGAIN)) { - - if (ret != NXT_ERROR) { - /* n == NXT_DECLINED: "\r" is not followed by "\n" */ - nxt_log(task, NXT_LOG_ERR, - "upstream sent invalid header line: \"%*s\\r...\"", - fs->u.header.parse.header_end - - fs->u.header.parse.header_name_start, - fs->u.header.parse.header_name_start); - } - - /* ret == NXT_ERROR */ - - nxt_fastcgi_source_fail(task, fs); - return; - } - - b = b->next; - - } while (b != NULL); -} - - -static void -nxt_fastcgi_source_sync_buffer(nxt_task_t *task, nxt_fastcgi_source_t *fs, - nxt_buf_t *b) -{ - if (nxt_buf_is_last(b)) { - nxt_log(task, NXT_LOG_ERR, "upstream closed prematurely connection"); - - } else { - nxt_log(task, NXT_LOG_ERR, "%ui buffers %uz each are not " - "enough to process upstream response header", - fs->upstream->buffers.max, fs->upstream->buffers.size); - } - - /* The stream source sends only the last and the nobuf sync buffer. */ - - nxt_fastcgi_source_fail(task, fs); -} - - -static nxt_int_t -nxt_fastcgi_source_header_process(nxt_task_t *task, nxt_fastcgi_source_t *fs) -{ - size_t len; - nxt_name_value_t *nv; - nxt_lvlhsh_query_t lhq; - nxt_http_header_parse_t *hp; - nxt_upstream_name_value_t *unv; - - hp = &fs->u.header.parse; - - len = hp->header_name_end - hp->header_name_start; - - if (len > 255) { - nxt_log(task, NXT_LOG_INFO, - "upstream sent too long header field name: \"%*s\"", - len, hp->header_name_start); - return NXT_ERROR; - } - - nv = nxt_list_add(fs->header_in.list); - if (nxt_slow_path(nv == NULL)) { - return NXT_ERROR; - } - - nv->hash = hp->header_hash; - nv->skip = 0; - nv->name_len = len; - nv->name_start = hp->header_name_start; - nv->value_len = hp->header_end - hp->header_start; - nv->value_start = hp->header_start; - - nxt_debug(task, "http header: \"%*s: %*s\"", - nv->name_len, nv->name_start, nv->value_len, nv->value_start); - - lhq.key_hash = nv->hash; - lhq.key.len = nv->name_len; - lhq.key.data = nv->name_start; - lhq.proto = &nxt_upstream_header_hash_proto; - - if (nxt_lvlhsh_find(&fs->header_in.hash, &lhq) == NXT_OK) { - unv = lhq.value; - - if (unv->handler(fs->upstream, nv) == NXT_OK) { - return NXT_ERROR; - } - } - - return NXT_OK; -} - - -static const nxt_upstream_name_value_t nxt_fastcgi_source_headers[] - nxt_aligned(32) = -{ - { nxt_fastcgi_source_status, - nxt_upstream_name_value("status") }, - - { nxt_fastcgi_source_content_length, - nxt_upstream_name_value("content-length") }, -}; - - -nxt_int_t -nxt_fastcgi_source_hash_create(nxt_mp_t *mp, nxt_lvlhsh_t *lh) -{ - return nxt_upstream_header_hash_add(mp, lh, nxt_fastcgi_source_headers, - nxt_nitems(nxt_fastcgi_source_headers)); -} - - -static nxt_int_t -nxt_fastcgi_source_status(nxt_upstream_source_t *us, nxt_name_value_t *nv) -{ - nxt_int_t n; - nxt_str_t s; - nxt_fastcgi_source_t *fs; - - s.len = nv->value_len; - s.data = nv->value_start; - - n = nxt_str_int_parse(&s); - - if (nxt_fast_path(n > 0)) { - fs = us->protocol_source; - fs->header_in.status = n; - return NXT_OK; - } - - return NXT_ERROR; -} - - -static nxt_int_t -nxt_fastcgi_source_content_length(nxt_upstream_source_t *us, - nxt_name_value_t *nv) -{ - nxt_off_t length; - nxt_fastcgi_source_t *fs; - - length = nxt_off_t_parse(nv->value_start, nv->value_len); - - if (nxt_fast_path(length > 0)) { - fs = us->protocol_source; - fs->header_in.content_length = length; - return NXT_OK; - } - - return NXT_ERROR; -} - - -static void -nxt_fastcgi_source_header_ready(nxt_fastcgi_source_t *fs, nxt_buf_t *b) -{ - /* - * Change the FastCGI source filter chain: - * stream source | FastCGI record filter | FastCGI body filter - */ - fs->record.next.filter = nxt_fastcgi_source_body_filter; - - if (nxt_buf_mem_used_size(&b->mem) != 0) { - fs->rest = b; - } - - if (fs->header_in.status == 0) { - /* The "200 OK" status by default. */ - fs->header_in.status = 200; - } - - fs->upstream->state->ready_handler(fs); -} - - -/* - * The FastCGI source body filter accumulates first body buffers before the next - * filter will be established and sets completion handler for the last buffer. - */ - -static void -nxt_fastcgi_source_body_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b, *in; - nxt_fastcgi_source_t *fs; - - fs = obj; - in = data; - - nxt_debug(task, "fastcgi source body filter"); - - for (b = in; b != NULL; b = b->next) { - - if (nxt_buf_is_last(b)) { - b->data = fs->upstream->data; - b->completion_handler = fs->upstream->state->completion_handler; - } - } - - if (fs->next != NULL) { - nxt_source_filter(task->thread, fs->upstream->work_queue, task, - fs->next, in); - return; - } - - nxt_buf_chain_add(&fs->rest, in); -} - - -static nxt_buf_t * -nxt_fastcgi_source_last_buf(nxt_fastcgi_parse_t *fp) -{ - nxt_buf_t *b; - nxt_fastcgi_source_t *fs; - - fs = fp->data; - - b = nxt_buf_sync_alloc(fp->mem_pool, NXT_BUF_SYNC_LAST); - - if (nxt_fast_path(b != NULL)) { - b->data = fs->upstream->data; - b->completion_handler = fs->upstream->state->completion_handler; - } - - return b; -} - - -static void -nxt_fastcgi_source_error(nxt_task_t *task, nxt_stream_source_t *stream) -{ - nxt_fastcgi_source_t *fs; - - nxt_thread_log_debug("fastcgi source error"); - - fs = stream->upstream->protocol_source; - - nxt_fastcgi_source_fail(task, fs); -} - - -static void -nxt_fastcgi_source_fail(nxt_task_t *task, nxt_fastcgi_source_t *fs) -{ - nxt_debug(task, "fastcgi source fail"); - - /* TODO: fail, next upstream, or bad gateway */ - - fs->upstream->state->error_handler(task, fs, NULL); -} diff --git a/src/nxt_fastcgi_source.h b/src/nxt_fastcgi_source.h deleted file mode 100644 index 979e962b..00000000 --- a/src/nxt_fastcgi_source.h +++ /dev/null @@ -1,93 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_FASTCGI_SOURCE_H_INCLUDED_ -#define _NXT_FASTCGI_SOURCE_H_INCLUDED_ - - -#define NXT_FASTCGI_BEGIN_REQUEST 1 -#define NXT_FASTCGI_ABORT_REQUEST 2 -#define NXT_FASTCGI_END_REQUEST 3 -#define NXT_FASTCGI_PARAMS 4 -#define NXT_FASTCGI_STDIN 5 -#define NXT_FASTCGI_STDOUT 6 -#define NXT_FASTCGI_STDERR 7 -#define NXT_FASTCGI_DATA 8 - - -typedef struct nxt_fastcgi_parse_s nxt_fastcgi_parse_t; - -struct nxt_fastcgi_parse_s { - u_char *pos; - - uint16_t length; /* 16 bits */ - uint8_t padding; - uint8_t type; - - uint8_t state; - uint8_t fastcgi_error; /* 1 bit */ - uint8_t error; /* 1 bit */ - uint8_t done; /* 1 bit */ - - /* FastCGI stdout and stderr buffer chains. */ - nxt_buf_t *out[2]; - - nxt_buf_t *(*last_buf)(nxt_fastcgi_parse_t *fp); - void *data; - nxt_mp_t *mem_pool; -}; - - -typedef struct { - nxt_fastcgi_parse_t parse; - nxt_source_hook_t next; -} nxt_fastcgi_source_record_t; - - -typedef struct { - nxt_str_t name; - nxt_str_t value; - uintptr_t data[3]; -} nxt_fastcgi_source_request_t; - - -typedef struct nxt_fastcgi_source_s nxt_fastcgi_source_t; -typedef nxt_int_t (*nxt_fastcgi_source_request_create_t)( - nxt_fastcgi_source_t *fs); - - -struct nxt_fastcgi_source_s { - nxt_source_hook_t query; - nxt_source_hook_t *next; - - nxt_upstream_source_t *upstream; - - nxt_fastcgi_source_request_create_t request_create; - - nxt_upstream_header_in_t header_in; - - nxt_buf_t *rest; - - uint32_t state; /* 2 bits */ - - nxt_fastcgi_source_record_t record; - - union { - nxt_fastcgi_source_request_t request; - } u; -}; - - -NXT_EXPORT void nxt_fastcgi_source_handler(nxt_task_t *task, - nxt_upstream_source_t *us, - nxt_fastcgi_source_request_create_t request_create); -NXT_EXPORT nxt_int_t nxt_fastcgi_source_hash_create(nxt_mp_t *mp, - nxt_lvlhsh_t *lh); -void nxt_fastcgi_record_parse(nxt_task_t *task, nxt_fastcgi_parse_t *fp, - nxt_buf_t *in); - - -#endif /* _NXT_FASTCGI_SOURCE_H_INCLUDED_ */ diff --git a/src/nxt_file.c b/src/nxt_file.c index a3fcda76..6f1a93e4 100644 --- a/src/nxt_file.c +++ b/src/nxt_file.c @@ -563,6 +563,25 @@ nxt_file_redirect(nxt_file_t *file, nxt_fd_t fd) } +/* nxt_file_stdout() redirects the stdout descriptor to the file. */ + +nxt_int_t +nxt_file_stdout(nxt_file_t *file) +{ + nxt_thread_log_debug("dup2(%FD, %FD, \"%FN\")", + file->fd, STDOUT_FILENO, file->name); + + if (dup2(file->fd, STDOUT_FILENO) != -1) { + return NXT_OK; + } + + nxt_thread_log_alert("dup2(%FD, %FD, \"%FN\") failed %E", + file->fd, STDOUT_FILENO, file->name, nxt_errno); + + return NXT_ERROR; +} + + /* nxt_file_stderr() redirects the stderr descriptor to the file. */ nxt_int_t diff --git a/src/nxt_file.h b/src/nxt_file.h index 945717b3..97636db6 100644 --- a/src/nxt_file.h +++ b/src/nxt_file.h @@ -191,6 +191,7 @@ NXT_EXPORT FILE *nxt_file_fopen(nxt_task_t *task, const char *pathname, NXT_EXPORT void nxt_file_fclose(nxt_task_t *task, FILE *fp); NXT_EXPORT nxt_int_t nxt_file_redirect(nxt_file_t *file, nxt_fd_t fd); +NXT_EXPORT nxt_int_t nxt_file_stdout(nxt_file_t *file); NXT_EXPORT nxt_int_t nxt_file_stderr(nxt_file_t *file); NXT_EXPORT nxt_int_t nxt_stderr_start(void); diff --git a/src/nxt_file_cache.c b/src/nxt_file_cache.c deleted file mode 100644 index 3af3c0c5..00000000 --- a/src/nxt_file_cache.c +++ /dev/null @@ -1,508 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -static nxt_int_t nxt_file_cache_lvlhsh_test(nxt_lvlhsh_key_t *hkey, void *data); -static nxt_work_handler_t nxt_file_cache_query_locked(nxt_file_cache_t *cache, - nxt_file_cache_query_t *q, nxt_lvlhsh_key_t *hkey); -static nxt_work_handler_t nxt_file_cache_node_hold(nxt_file_cache_t *cache, - nxt_file_cache_query_t *q, nxt_lvlhsh_key_t *hkey); -static nxt_work_handler_t nxt_file_cache_node_test(nxt_file_cache_t *cache, - nxt_file_cache_query_t *q); - -static void nxt_file_cache_wait_handler(void *data); -static void nxt_file_cache_timeout_handler(nxt_event_timer_t *ev); -static void nxt_file_cache_wake_handler(void *data); - -static nxt_file_cache_node_t *nxt_file_cache_node_alloc(nxt_cache_t *cache); -static void nxt_file_cache_node_free(nxt_file_cache_t *cache, - nxt_file_cache_node_t *node, nxt_bool_t fast); -static nxt_file_cache_query_wait_t *nxt_file_cache_query_wait_alloc( - nxt_file_cache_t *cache, nxt_bool_t *fast); -static void nxt_file_cache_query_wait_free(nxt_file_cache_t *cache, - nxt_file_cache_query_wait_t *qw); -static void nxt_file_cache_lock(nxt_file_cache_t *cache); -static void nxt_file_cache_unlock(nxt_file_cache_t *cache); - - -void -nxt_file_cache_init(nxt_cache_t *cache) -{ - static const nxt_lvlhsh_ctx_t ctx = { - nxt_file_cache_lvlhsh_test, - nxt_lvlhsh_alloc, - nxt_lvlhsh_free, - 0, - }; - - /* lvlhsh with large first level. */ - cache->lvlhsh.shift[1] = 10; - - cache->lvlhsh.ctx = &ctx; - - cache->start_time = nxt_thread_time(); -} - - -static nxt_int_t -nxt_file_cache_lvlhsh_test(nxt_lvlhsh_key_t *hkey, void *data) -{ - nxt_file_cache_node_t *node; - - node = data; - - if (nxt_strmem_eq(&hkey->key, node->key_data, node->key_len)) { - return NXT_OK; - } - - return NXT_DECLINED; -} - - -void -nxt_file_cache_query(nxt_file_cache_t *cache, nxt_file_cache_query_t *q) -{ - nxt_lvlhsh_key_t hkey; - nxt_work_handler_t handler; - - if (cache != NULL) { - hkey.key.len = q->key_len; - hkey.key.data = q->key_data; - hkey.key_hash = nxt_murmur_hash2(q->key_data, q->key_len); - hkey.replace = 0; - - nxt_file_cache_lock(cache); - - handler = nxt_file_cache_query_locked(cache, q, &hkey); - - nxt_file_cache_unlock(cache); - - } else { - handler = q->state->nocache_handler; - } - - handler(q); -} - - -static nxt_work_handler_t -nxt_file_cache_query_locked(nxt_file_cache_t *cache, nxt_file_cache_query_t *q, - nxt_lvlhsh_key_t *hkey) -{ - nxt_int_t ret; - nxt_bool_t fast; - nxt_work_handler_t handler; - nxt_file_cache_node_t *node, *sentinel; - nxt_file_cache_query_wait_t *qw; - nxt_file_cache_query_state_t *state; - - state = q->state; - sentinel = nxt_file_cache_node_alloc(cache); - - if (nxt_slow_path(sentinel == NULL)) { - return state->error_handler; - } - - sentinel->key_data = q->key_data; - sentinel->key_len = q->key_len; - hkey->value = sentinel; - - /* - * Try to insert an empty sentinel node to hold updating - * process if there is no existent cache node in cache. - */ - - ret = nxt_lvlhsh_insert(&cache->lvlhsh, hkey); - - if (ret == NXT_OK) { - /* The sentinel node was successully added. */ - - q->node = sentinel; - sentinel->updating = 1; - return state->update_handler; - } - - nxt_cache_node_free(cache, sentinel, 1); - - if (ret == NXT_ERROR) { - return state->error_handler; - } - - /* NXT_DECLINED: a cache node exists. */ - - node = hkey->value; - node->count++; - q->node = node; - - handler = nxt_cache_node_test(cache, q); - - if (handler == NULL) { - /* Add the node to a wait queue. */ - - qw = nxt_cache_query_wait_alloc(cache, &fast); - if (nxt_slow_path(qw == NULL)) { - return state->error_handler; - } - - if (!fast) { - /* The node state may be changed during slow allocation. */ - handler = nxt_cache_node_test(cache, q); - - if (handler != NULL) { - nxt_cache_query_wait_free(cache, qw); - return handler; - } - } - - qw->query = q; - qw->next = node->waiting; - qw->busy = 0; - qw->deleted = 0; - qw->pid = nxt_pid; - qw->engine = nxt_thread_event_engine(); - qw->handler = nxt_cache_wake_handler; - qw->cache = cache; - - node->waiting = qw; - - return nxt_cache_wait_handler; - } - - return handler; -} - - -static nxt_work_handler_t -nxt_cache_node_test(nxt_cache_t *cache, nxt_cache_query_t *q) -{ - nxt_time_t expiry; - nxt_cache_node_t *node; - nxt_cache_query_state_t *state; - - q->stale = 0; - state = q->state; - node = q->node; - - expiry = cache->start_time + node->expiry; - - if (nxt_thread_time() < expiry) { - return state->ready_handler; - } - - /* - * A valid stale or empty sentinel cache node. - * The sentinel node can be only in updating state. - */ - - if (node->updating) { - - if (node->expiry != 0) { - /* A valid stale cache node. */ - - q->stale = 1; - - if (q->use_stale) { - return state->stale_handler; - } - } - - return NULL; - } - - /* A valid stale cache node is not being updated now. */ - - q->stale = 1; - - if (q->use_stale) { - - if (q->update_stale) { - node->updating = 1; - return state->update_stale_handler; - } - - return state->stale_handler; - } - - node->updating = 1; - return state->update_handler; -} - - -static void -nxt_cache_wait_handler(void *data) -{ - nxt_thread_t *thr; - nxt_event_timer_t *ev; - nxt_cache_query_t *q; - - q = data; - - if (&q->timeout == 0) { - return; - } - - ev = &q->timer; - - if (!nxt_event_timer_is_set(ev)) { - thr = nxt_thread(); - ev->log = thr->log; - ev->handler = nxt_cache_timeout_handler; - ev->data = q; - nxt_event_timer_ident(ev, -1); - - nxt_event_timer_add(thr->engine, ev, q->timeout); - } -} - - -static void -nxt_cache_timeout_handler(nxt_event_timer_t *ev) -{ - nxt_cache_query_t *q; - - q = ev->data; - - q->state->timeout_handler(q); -} - - -static void -nxt_cache_wake_handler(void *data) -{ - nxt_cache_t *cache; - nxt_work_handler_t handler; - nxt_cache_query_t *q; - nxt_cache_query_wait_t *qw; - - qw = data; - q = qw->query; - cache = qw->cache; - - nxt_cache_lock(cache); - - handler = nxt_cache_node_test(cache, q); - - if (handler == NULL) { - /* Wait again. */ - qw->next = q->node->waiting; - q->node->waiting = qw; - } - - nxt_cache_unlock(cache); - - if (handler != NULL) { - nxt_cache_query_wait_free(cache, qw); - } - - handler(q); -} - - -static nxt_cache_node_t * -nxt_cache_node_alloc(nxt_cache_t *cache) -{ - nxt_queue_node_t *qn; - nxt_cache_node_t *node; - - qn = nxt_queue_first(&cache->free_nodes); - - if (nxt_fast_path(qn != nxt_queue_tail(&cache->free_nodes))) { - cache->nfree_nodes--; - nxt_queue_remove(qn); - - node = nxt_queue_node_data(qn, nxt_cache_node_t, queue); - nxt_memzero(node, sizeof(nxt_cache_node_t)); - - return node; - } - - nxt_cache_unlock(cache); - - node = cache->alloc(cache->data, sizeof(nxt_cache_node_t)); - - nxt_cache_lock(cache); - - return node; -} - - -static void -nxt_cache_node_free(nxt_cache_t *cache, nxt_cache_node_t *node, nxt_bool_t fast) -{ - if (fast || cache->nfree_nodes < 32) { - nxt_queue_insert_head(&cache->free_nodes, &node->queue); - cache->nfree_nodes++; - return; - } - - nxt_cache_unlock(cache); - - cache->free(cache->data, node); - - nxt_cache_lock(cache); -} - - -static nxt_cache_query_wait_t * -nxt_cache_query_wait_alloc(nxt_cache_t *cache, nxt_bool_t *fast) -{ - nxt_cache_query_wait_t *qw; - - qw = cache->free_query_wait; - - if (nxt_fast_path(qw != NULL)) { - cache->free_query_wait = qw->next; - cache->nfree_query_wait--; - - *fast = 1; - return qw; - } - - nxt_cache_unlock(cache); - - qw = cache->alloc(cache->data, sizeof(nxt_cache_query_wait_t)); - *fast = 0; - - nxt_cache_lock(cache); - - return qw; -} - - -static void -nxt_cache_query_wait_free(nxt_cache_t *cache, nxt_cache_query_wait_t *qw) -{ - if (cache->nfree_query_wait < 32) { - qw->next = cache->free_query_wait; - cache->free_query_wait = qw; - cache->nfree_query_wait++; - return; - } - - nxt_cache_unlock(cache); - - cache->free(cache->data, qw); - - nxt_cache_lock(cache); -} - - -#if 0 - -nxt_int_t -nxt_cache_update(nxt_cache_t *cache, nxt_cache_node_t *node) -{ - nxt_lvlhsh_key_t hkey; - - if (node->expiry == 0) { - /* An empty sentinel node. */ - nxt_cache_release(cache, node); - return; - } - - hkey.key.len = node->key_len; - hkey.key.data = node->key_data; - hkey.key_hash = nxt_murmur_hash2(node->key_data, node->key_len); - hkey.replace = 1; - hkey.value = node; - - node->count = 1; - - if (nxt_lvlhsh_insert(&cache->lvlhsh, &hkey) != NXT_OK) { - return NXT_ERROR; - } - - node = hkey.value; - - if (node != NULL) { - if (node->count != 0) { - node->delete = 1; - - } else { - // delete cache node - } - } - - return NXT_OK; -} - -#endif - - -void -nxt_cache_node_release(nxt_cache_t *cache, nxt_cache_node_t *node) -{ - nxt_bool_t delete; - - nxt_cache_lock(cache); - - delete = nxt_cache_node_release_locked(cache, node); - - nxt_cache_unlock(cache); - - if (delete) { - nxt_thread_work_queue_add(cache->delete_handler, node); - } -} - - -nxt_bool_t -nxt_cache_node_release_locked(nxt_cache_t *cache, nxt_cache_node_t *node) -{ -#if 0 - nxt_lvlhsh_key_t hkey; -#endif - - node->count--; - - if (node->count != 0) { - return 0; - } - - if (!node->deleted) { - /* - * A cache node is locked whilst its count is non zero. - * To minimize number of operations the node's place in expiry - * queue can be updated only if the node is not currently used. - */ - node->accessed = nxt_thread_time() - cache->start_time; - - nxt_queue_remove(&node->queue); - nxt_queue_insert_head(&cache->expiry_queue, &node->queue); - - return 0; - } - -#if 0 - hkey.key.len = node->key_len; - hkey.key.data = node->key_data; - hkey.key_hash = nxt_murmur_hash2(node->key_data, node->key_len); - - nxt_lvlhsh_delete(&cache->lvlhsh, &hkey); -#endif - - return 1; -} - - -static void -nxt_file_cache_lock(nxt_file_cache_t *cache) -{ - if (cache->shared) { - nxt_thread_spin_lock(&cache->lock); - } -} - - -static void -nxt_file_cache_unlock(nxt_file_cache_t *cache) -{ - if (cache->shared) { - nxt_thread_spin_unlock(&cache->lock); - } -} diff --git a/src/nxt_fs.c b/src/nxt_fs.c index a467da98..e10c5bcb 100644 --- a/src/nxt_fs.c +++ b/src/nxt_fs.c @@ -20,7 +20,7 @@ nxt_fs_mkdir_all(const u_char *dir, mode_t mode) nxt_assert(dirlen < PATH_MAX && dirlen > 1 && dir[0] == '/'); dst = path; - start = end = (char *) dir; + start = (char *) dir; while (*start != '\0') { if (*start == '/') { diff --git a/src/nxt_h1proto.c b/src/nxt_h1proto.c index 1e37273f..df1f82f9 100644 --- a/src/nxt_h1proto.c +++ b/src/nxt_h1proto.c @@ -507,6 +507,7 @@ nxt_h1p_conn_request_init(nxt_task_t *task, void *obj, void *data) r->conf = joint; skcf = joint->socket_conf; + r->log_route = skcf->log_route; if (c->local == NULL) { c->local = skcf->sockaddr; @@ -576,6 +577,15 @@ nxt_h1p_conn_request_header_parse(nxt_task_t *task, void *obj, void *data) */ h1p->keepalive = (h1p->parser.version.s.minor != '0'); + r->request_line.start = h1p->parser.method.start; + r->request_line.length = h1p->parser.request_line_end + - r->request_line.start; + + if (nxt_slow_path(r->log_route)) { + nxt_log(task, NXT_LOG_NOTICE, "http request line \"%V\"", + &r->request_line); + } + ret = nxt_h1p_header_process(task, h1p, r); if (nxt_fast_path(ret == NXT_OK)) { diff --git a/src/nxt_http.h b/src/nxt_http.h index a8725d9f..08e1fcbe 100644 --- a/src/nxt_http.h +++ b/src/nxt_http.h @@ -144,6 +144,7 @@ struct nxt_http_request_s { nxt_str_t host; nxt_str_t server_name; + nxt_str_t request_line; nxt_str_t target; nxt_str_t version; nxt_str_t *method; @@ -189,6 +190,8 @@ struct nxt_http_request_s { nxt_http_status_t status:16; + uint8_t log_route; /* 1 bit */ + uint8_t pass_count; /* 8 bits */ uint8_t app_target; nxt_http_protocol_t protocol:8; /* 2 bits */ @@ -223,6 +226,7 @@ typedef struct nxt_http_route_addr_rule_s nxt_http_route_addr_rule_t; typedef struct { + nxt_conf_value_t *rewrite; nxt_conf_value_t *pass; nxt_conf_value_t *ret; nxt_conf_value_t *location; @@ -250,6 +254,7 @@ struct nxt_http_action_s { nxt_str_t *pass; } u; + nxt_tstr_t *rewrite; nxt_http_action_t *fallback; }; @@ -375,6 +380,11 @@ nxt_int_t nxt_upstreams_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_int_t nxt_upstreams_joint_create(nxt_router_temp_conf_t *tmcf, nxt_upstream_t ***upstream_joint); +nxt_int_t nxt_http_rewrite_init(nxt_router_conf_t *rtcf, + nxt_http_action_t *action, nxt_http_action_conf_t *acf); +nxt_int_t nxt_http_rewrite(nxt_task_t *task, nxt_http_request_t *r, + nxt_http_action_t *action); + nxt_int_t nxt_http_return_init(nxt_router_conf_t *rtcf, nxt_http_action_t *action, nxt_http_action_conf_t *acf); diff --git a/src/nxt_http_js.c b/src/nxt_http_js.c index 5a08a309..72ba761f 100644 --- a/src/nxt_http_js.c +++ b/src/nxt_http_js.c @@ -15,15 +15,19 @@ static njs_int_t nxt_http_js_ext_host(njs_vm_t *vm, njs_object_prop_t *prop, static njs_int_t nxt_http_js_ext_remote_addr(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); -static njs_int_t nxt_http_js_ext_get_arg(njs_vm_t *vm, +static njs_int_t nxt_http_js_ext_get_args(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); static njs_int_t nxt_http_js_ext_get_header(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); +static njs_int_t nxt_http_js_ext_keys_header(njs_vm_t *vm, + njs_value_t *value, njs_value_t *keys); static njs_int_t nxt_http_js_ext_get_cookie(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); +static njs_int_t nxt_http_js_ext_keys_cookie(njs_vm_t *vm, njs_value_t *value, + njs_value_t *keys); static njs_external_t nxt_http_js_proto[] = { @@ -55,12 +59,11 @@ static njs_external_t nxt_http_js_proto[] = { }, { - .flags = NJS_EXTERN_OBJECT, + .flags = NJS_EXTERN_PROPERTY, .name.string = njs_str("args"), .enumerable = 1, - .u.object = { - .enumerable = 1, - .prop_handler = nxt_http_js_ext_get_arg, + .u.property = { + .handler = nxt_http_js_ext_get_args, } }, @@ -71,6 +74,7 @@ static njs_external_t nxt_http_js_proto[] = { .u.object = { .enumerable = 1, .prop_handler = nxt_http_js_ext_get_header, + .keys = nxt_http_js_ext_keys_header, } }, @@ -81,6 +85,7 @@ static njs_external_t nxt_http_js_proto[] = { .u.object = { .enumerable = 1, .prop_handler = nxt_http_js_ext_get_cookie, + .keys = nxt_http_js_ext_keys_cookie, } }, }; @@ -144,14 +149,13 @@ nxt_http_js_ext_remote_addr(njs_vm_t *vm, njs_object_prop_t *prop, static njs_int_t -nxt_http_js_ext_get_arg(njs_vm_t *vm, njs_object_prop_t *prop, +nxt_http_js_ext_get_args(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { - njs_int_t rc; - njs_str_t key; - nxt_array_t *args; - nxt_http_request_t *r; - nxt_http_name_value_t *nv, *start, *end; + njs_int_t ret; + njs_value_t *args; + njs_opaque_value_t val; + nxt_http_request_t *r; r = njs_vm_external(vm, nxt_js_proto_id, value); if (r == NULL) { @@ -159,33 +163,18 @@ nxt_http_js_ext_get_arg(njs_vm_t *vm, njs_object_prop_t *prop, return NJS_DECLINED; } - rc = njs_vm_prop_name(vm, prop, &key); - if (rc != NJS_OK) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - args = nxt_http_arguments_parse(r); - if (nxt_slow_path(args == NULL)) { - return NJS_ERROR; - } - - start = args->elts; - end = start + args->nelts; + args = njs_value_arg(&val); - for (nv = start; nv < end; nv++) { + ret = njs_vm_query_string_parse(vm, r->args->start, + r->args->start + r->args->length, args); - if (key.length == nv->name_length - && memcmp(key.start, nv->name, nv->name_length) == 0) - { - return njs_vm_value_string_set(vm, retval, nv->value, - nv->value_length); - } + if (ret == NJS_ERROR) { + return NJS_ERROR; } - njs_value_undefined_set(retval); + njs_value_assign(retval, args); - return NJS_DECLINED; + return NJS_OK; } @@ -228,6 +217,41 @@ nxt_http_js_ext_get_header(njs_vm_t *vm, njs_object_prop_t *prop, static njs_int_t +nxt_http_js_ext_keys_header(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) +{ + njs_int_t rc; + nxt_http_field_t *f; + nxt_http_request_t *r; + + rc = njs_vm_array_alloc(vm, keys, 4); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + r = njs_vm_external(vm, nxt_js_proto_id, value); + if (r == NULL) { + return NJS_OK; + } + + nxt_list_each(f, r->fields) { + + value = njs_vm_array_push(vm, keys); + if (value == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, value, f->name, f->name_length); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + } nxt_list_loop; + + return NJS_OK; +} + + +static njs_int_t nxt_http_js_ext_get_cookie(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { @@ -271,3 +295,46 @@ nxt_http_js_ext_get_cookie(njs_vm_t *vm, njs_object_prop_t *prop, return NJS_DECLINED; } + + +static njs_int_t +nxt_http_js_ext_keys_cookie(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) +{ + njs_int_t rc; + nxt_array_t *cookies; + nxt_http_request_t *r; + nxt_http_name_value_t *nv, *start, *end; + + rc = njs_vm_array_alloc(vm, keys, 4); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + r = njs_vm_external(vm, nxt_js_proto_id, value); + if (r == NULL) { + return NJS_OK; + } + + cookies = nxt_http_cookies_parse(r); + if (nxt_slow_path(cookies == NULL)) { + return NJS_ERROR; + } + + start = cookies->elts; + end = start + cookies->nelts; + + for (nv = start; nv < end; nv++) { + + value = njs_vm_array_push(vm, keys); + if (value == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, value, nv->name, nv->name_length); + if (rc != NJS_OK) { + return NJS_ERROR; + } + } + + return NJS_OK; +} diff --git a/src/nxt_http_parse.c b/src/nxt_http_parse.c index f39d8f67..50cbda2b 100644 --- a/src/nxt_http_parse.c +++ b/src/nxt_http_parse.c @@ -19,8 +19,6 @@ static u_char *nxt_http_lookup_field_end(u_char *p, const u_char *end); static nxt_int_t nxt_http_parse_field_end(nxt_http_request_parse_t *rp, u_char **pos, const u_char *end); -static nxt_int_t nxt_http_parse_complex_target(nxt_http_request_parse_t *rp); - static nxt_int_t nxt_http_field_hash_test(nxt_lvlhsh_query_t *lhq, void *data); static nxt_int_t nxt_http_field_hash_collision(nxt_lvlhsh_query_t *lhq, @@ -417,23 +415,25 @@ space_after_target: { rp->version.ui64 = ver.ui64; - if (nxt_fast_path(p[9] == '\r')) { - p += 10; + p += 9; + if (nxt_fast_path(*p == '\r')) { - if (nxt_slow_path(p == end)) { + if (nxt_slow_path(p + 1 == end)) { return NXT_AGAIN; } - if (nxt_slow_path(*p != '\n')) { + if (nxt_slow_path(p[1] != '\n')) { return NXT_HTTP_PARSE_INVALID; } - *pos = p + 1; + *pos = p + 2; } else { - *pos = p + 10; + *pos = p + 1; } + rp->request_line_end = p; + if (rp->complex_target != 0 #if 0 || rp->quoted_target != 0 @@ -852,7 +852,7 @@ static const uint8_t nxt_http_normal[32] nxt_aligned(32) = { }; -static nxt_int_t +nxt_int_t nxt_http_parse_complex_target(nxt_http_request_parse_t *rp) { u_char *p, *u, c, ch, high, *args; diff --git a/src/nxt_http_parse.h b/src/nxt_http_parse.h index 2b714464..fa95e842 100644 --- a/src/nxt_http_parse.h +++ b/src/nxt_http_parse.h @@ -41,6 +41,7 @@ struct nxt_http_request_parse_s { u_char *target_start; u_char *target_end; + u_char *request_line_end; nxt_str_t path; nxt_str_t args; @@ -126,6 +127,7 @@ nxt_uint_t nxt_http_fields_hash_collisions(nxt_lvlhsh_t *hash, nxt_int_t nxt_http_fields_process(nxt_list_t *fields, nxt_lvlhsh_t *hash, void *ctx); +nxt_int_t nxt_http_parse_complex_target(nxt_http_request_parse_t *rp); nxt_buf_t *nxt_http_chunk_parse(nxt_task_t *task, nxt_http_chunk_parse_t *hcp, nxt_buf_t *in); diff --git a/src/nxt_http_request.c b/src/nxt_http_request.c index e78975aa..48f7dbe3 100644 --- a/src/nxt_http_request.c +++ b/src/nxt_http_request.c @@ -555,9 +555,18 @@ void nxt_http_request_action(nxt_task_t *task, nxt_http_request_t *r, nxt_http_action_t *action) { + nxt_int_t ret; + if (nxt_fast_path(action != NULL)) { do { + if (action->rewrite != NULL) { + ret = nxt_http_rewrite(task, r, action); + if (nxt_slow_path(ret != NXT_OK)) { + break; + } + } + action = action->handler(task, r, action); if (action == NULL) { @@ -622,8 +631,9 @@ void nxt_http_request_header_send(nxt_task_t *task, nxt_http_request_t *r, nxt_work_handler_t body_handler, void *data) { - u_char *p, *end; - nxt_http_field_t *server, *date, *content_length; + u_char *p, *end, *server_string; + nxt_http_field_t *server, *date, *content_length; + nxt_socket_conf_t *skcf; /* * TODO: "Server", "Date", and "Content-Length" processing should be moved @@ -635,7 +645,12 @@ nxt_http_request_header_send(nxt_task_t *task, nxt_http_request_t *r, goto fail; } - nxt_http_field_set(server, "Server", NXT_SERVER); + skcf = r->conf->socket_conf; + server_string = (u_char *) (skcf->server_version ? NXT_SERVER : NXT_NAME); + + nxt_http_field_name_set(server, "Server"); + server->value = server_string; + server->value_length = nxt_strlen(server_string); if (r->resp.date == NULL) { date = nxt_list_zero_add(r->resp.fields); diff --git a/src/nxt_http_rewrite.c b/src/nxt_http_rewrite.c new file mode 100644 index 00000000..b800a919 --- /dev/null +++ b/src/nxt_http_rewrite.c @@ -0,0 +1,104 @@ + +/* + * Copyright (C) Zhidao HONG + * Copyright (C) NGINX, Inc. + */ + +#include <nxt_router.h> +#include <nxt_http.h> + + +nxt_int_t +nxt_http_rewrite_init(nxt_router_conf_t *rtcf, nxt_http_action_t *action, + nxt_http_action_conf_t *acf) + { + nxt_str_t str; + + nxt_conf_get_string(acf->rewrite, &str); + + action->rewrite = nxt_tstr_compile(rtcf->tstr_state, &str, 0); + if (nxt_slow_path(action->rewrite == NULL)) { + return NXT_ERROR; + } + + return NXT_OK; +} + + +nxt_int_t +nxt_http_rewrite(nxt_task_t *task, nxt_http_request_t *r, + nxt_http_action_t *action) +{ + u_char *p; + nxt_int_t ret; + nxt_str_t str, encoded_path, target; + nxt_router_conf_t *rtcf; + nxt_http_request_parse_t rp; + + if (nxt_tstr_is_const(action->rewrite)) { + nxt_tstr_str(action->rewrite, &str); + + } else { + rtcf = r->conf->socket_conf->router_conf; + + ret = nxt_tstr_query_init(&r->tstr_query, rtcf->tstr_state, + &r->tstr_cache, r, r->mem_pool); + if (nxt_slow_path(ret != NXT_OK)) { + return NXT_ERROR; + } + + nxt_tstr_query(task, r->tstr_query, action->rewrite, &str); + + if (nxt_slow_path(nxt_tstr_query_failed(r->tstr_query))) { + return NXT_ERROR; + } + } + + nxt_memzero(&rp, sizeof(nxt_http_request_parse_t)); + + rp.mem_pool = r->mem_pool; + + rp.target_start = str.start; + rp.target_end = str.start + str.length; + + ret = nxt_http_parse_complex_target(&rp); + if (nxt_slow_path(ret != NXT_OK)) { + return NXT_ERROR; + } + + p = (rp.args.length > 0) ? rp.args.start - 1 : rp.target_end; + + encoded_path.start = rp.target_start; + encoded_path.length = p - encoded_path.start; + + if (r->args->length == 0) { + r->target = encoded_path; + + } else { + target.length = encoded_path.length + 1 + r->args->length; + + target.start = nxt_mp_alloc(r->mem_pool, target.length); + if (target.start == NULL) { + return NXT_ERROR; + } + + p = nxt_cpymem(target.start, encoded_path.start, encoded_path.length); + *p++ = '?'; + nxt_memcpy(p, r->args->start, r->args->length); + + r->target = target; + } + + r->path = nxt_mp_alloc(r->mem_pool, sizeof(nxt_str_t)); + if (nxt_slow_path(r->path == NULL)) { + return NXT_ERROR; + } + + *r->path = rp.path; + + if (nxt_slow_path(r->log_route)) { + nxt_log(task, NXT_LOG_NOTICE, "URI rewritten to \"%V\"", &r->target); + } + + return NXT_OK; +} diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 7081ff7e..0935dd4a 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -579,6 +579,11 @@ nxt_http_route_match_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, static nxt_conf_map_t nxt_http_route_action_conf[] = { { + nxt_string("rewrite"), + NXT_CONF_MAP_PTR, + offsetof(nxt_http_action_conf_t, rewrite) + }, + { nxt_string("pass"), NXT_CONF_MAP_PTR, offsetof(nxt_http_action_conf_t, pass) @@ -659,6 +664,13 @@ nxt_http_action_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, rtcf = tmcf->router_conf; mp = rtcf->mem_pool; + if (acf.rewrite != NULL) { + ret = nxt_http_rewrite_init(rtcf, action, &acf); + if (nxt_slow_path(ret != NXT_OK)) { + return ret; + } + } + if (acf.ret != NULL) { return nxt_http_return_init(rtcf, action, &acf); } @@ -1312,8 +1324,8 @@ nxt_http_pass_var(nxt_task_t *task, nxt_http_request_t *r, goto fail; } - action = nxt_mp_get(r->mem_pool, - sizeof(nxt_http_action_t) + sizeof(nxt_str_t)); + action = nxt_mp_zget(r->mem_pool, + sizeof(nxt_http_action_t) + sizeof(nxt_str_t)); if (nxt_slow_path(action == NULL)) { goto fail; } @@ -1496,7 +1508,7 @@ nxt_http_action_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, rtcf = tmcf->router_conf; mp = rtcf->mem_pool; - action = nxt_mp_alloc(mp, sizeof(nxt_http_action_t)); + action = nxt_mp_zalloc(mp, sizeof(nxt_http_action_t)); if (nxt_slow_path(action == NULL)) { return NULL; } @@ -1525,7 +1537,7 @@ nxt_http_pass_application(nxt_task_t *task, nxt_router_conf_t *rtcf, { nxt_http_action_t *action; - action = nxt_mp_alloc(rtcf->mem_pool, sizeof(nxt_http_action_t)); + action = nxt_mp_zalloc(rtcf->mem_pool, sizeof(nxt_http_action_t)); if (nxt_slow_path(action == NULL)) { return NULL; } @@ -1540,21 +1552,29 @@ static nxt_http_action_t * nxt_http_route_handler(nxt_task_t *task, nxt_http_request_t *r, nxt_http_action_t *start) { + size_t i; nxt_http_route_t *route; nxt_http_action_t *action; - nxt_http_route_match_t **match, **end; route = start->u.route; - match = &route->match[0]; - end = match + route->items; - while (match < end) { - action = nxt_http_route_match(task, r, *match); + for (i = 0; i < route->items; i++) { + action = nxt_http_route_match(task, r, route->match[i]); + + if (nxt_slow_path(r->log_route)) { + uint32_t lvl = (action == NULL) ? NXT_LOG_INFO : NXT_LOG_NOTICE; + const char *sel = (action == NULL) ? "discarded" : "selected"; + + if (route->name.length == 0) { + nxt_log(task, lvl, "\"routes/%z\" %s", i, sel); + } else { + nxt_log(task, lvl, "\"routes/%V/%z\" %s", &route->name, i, sel); + } + } + if (action != NULL) { return action; } - - match++; } nxt_http_request_error(task, r, NXT_HTTP_NOT_FOUND); diff --git a/src/nxt_http_source.c b/src/nxt_http_source.c deleted file mode 100644 index 889dcd08..00000000 --- a/src/nxt_http_source.c +++ /dev/null @@ -1,629 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -typedef struct { - nxt_http_chunk_parse_t parse; - nxt_source_hook_t next; -} nxt_http_source_chunk_t; - - -static nxt_buf_t *nxt_http_source_request_create(nxt_http_source_t *hs); - -static void nxt_http_source_status_filter(nxt_task_t *task, void *obj, - void *data); -static void nxt_http_source_header_filter(nxt_task_t *task, void *obj, - void *data); - -static nxt_int_t nxt_http_source_header_line_process(nxt_http_source_t *hs); -static nxt_int_t nxt_http_source_content_length(nxt_upstream_source_t *us, - nxt_name_value_t *nv); -static nxt_int_t nxt_http_source_transfer_encoding(nxt_upstream_source_t *us, - nxt_name_value_t *nv); - -static void nxt_http_source_header_ready(nxt_task_t *task, - nxt_http_source_t *hs, nxt_buf_t *rest); -static void nxt_http_source_chunk_filter(nxt_task_t *task, void *obj, - void *data); -static void nxt_http_source_chunk_error(nxt_task_t *task, void *obj, - void *data); -static void nxt_http_source_body_filter(nxt_task_t *task, void *obj, - void *data); - -static void nxt_http_source_sync_buffer(nxt_task_t *task, nxt_http_source_t *hs, - nxt_buf_t *b); -static void nxt_http_source_error(nxt_task_t *task, - nxt_stream_source_t *stream); -static void nxt_http_source_fail(nxt_task_t *task, nxt_http_source_t *hs); -static void nxt_http_source_message(const char *msg, size_t len, u_char *p); - - -void -nxt_http_source_handler(nxt_task_t *task, nxt_upstream_source_t *us, - nxt_http_source_request_create_t request_create) -{ - nxt_http_source_t *hs; - nxt_stream_source_t *stream; - - hs = nxt_mp_zget(us->buffers.mem_pool, sizeof(nxt_http_source_t)); - if (nxt_slow_path(hs == NULL)) { - goto fail; - } - - us->protocol_source = hs; - - hs->header_in.list = nxt_list_create(us->buffers.mem_pool, 8, - sizeof(nxt_name_value_t)); - if (nxt_slow_path(hs->header_in.list == NULL)) { - goto fail; - } - - hs->header_in.hash = us->header_hash; - hs->upstream = us; - hs->request_create = request_create; - - stream = us->stream; - - if (stream == NULL) { - stream = nxt_mp_zget(us->buffers.mem_pool, sizeof(nxt_stream_source_t)); - if (nxt_slow_path(stream == NULL)) { - goto fail; - } - - us->stream = stream; - stream->upstream = us; - - } else { - nxt_memzero(stream, sizeof(nxt_stream_source_t)); - } - - /* - * Create the HTTP source filter chain: - * stream source | HTTP status line filter - */ - stream->next = &hs->query; - stream->error_handler = nxt_http_source_error; - - hs->query.context = hs; - hs->query.filter = nxt_http_source_status_filter; - - hs->header_in.content_length = -1; - - stream->out = nxt_http_source_request_create(hs); - - if (nxt_fast_path(stream->out != NULL)) { - nxt_memzero(&hs->u.status_parse, sizeof(nxt_http_status_parse_t)); - - nxt_stream_source_connect(task, stream); - return; - } - -fail: - - nxt_http_source_fail(task, hs); -} - - -nxt_inline u_char * -nxt_http_source_copy(u_char *p, nxt_str_t *src, size_t len) -{ - u_char *s; - - if (nxt_fast_path(len >= src->len)) { - len = src->len; - src->len = 0; - - } else { - src->len -= len; - } - - s = src->data; - src->data += len; - - return nxt_cpymem(p, s, len); -} - - -static nxt_buf_t * -nxt_http_source_request_create(nxt_http_source_t *hs) -{ - nxt_int_t ret; - nxt_buf_t *b, *req, **prev; - - nxt_thread_log_debug("http source create request"); - - prev = &req; - -new_buffer: - - ret = nxt_buf_pool_mem_alloc(&hs->upstream->buffers, 0); - if (nxt_slow_path(ret != NXT_OK)) { - return NULL; - } - - b = hs->upstream->buffers.current; - hs->upstream->buffers.current = NULL; - - *prev = b; - prev = &b->next; - - for ( ;; ) { - ret = hs->request_create(hs); - - if (nxt_fast_path(ret == NXT_OK)) { - b->mem.free = nxt_http_source_copy(b->mem.free, &hs->u.request.copy, - b->mem.end - b->mem.free); - - if (nxt_fast_path(hs->u.request.copy.len == 0)) { - continue; - } - - nxt_thread_log_debug("\"%*s\"", b->mem.free - b->mem.pos, - b->mem.pos); - - goto new_buffer; - } - - if (nxt_slow_path(ret == NXT_ERROR)) { - return NULL; - } - - /* ret == NXT_DONE */ - break; - } - - nxt_thread_log_debug("\"%*s\"", b->mem.free - b->mem.pos, b->mem.pos); - - return req; -} - - -static void -nxt_http_source_status_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_int_t ret; - nxt_buf_t *b; - nxt_http_source_t *hs; - - hs = obj; - b = data; - - /* - * No cycle over buffer chain is required since at - * start the stream source passes buffers one at a time. - */ - - nxt_debug(task, "http source status filter"); - - if (nxt_slow_path(nxt_buf_is_sync(b))) { - nxt_http_source_sync_buffer(task, hs, b); - return; - } - - ret = nxt_http_status_parse(&hs->u.status_parse, &b->mem); - - if (nxt_fast_path(ret == NXT_OK)) { - /* - * Change the HTTP source filter chain: - * stream source | HTTP header filter - */ - hs->query.filter = nxt_http_source_header_filter; - - nxt_debug(task, "upstream status: \"%*s\"", - hs->u.status_parse.end - b->mem.start, b->mem.start); - - hs->header_in.status = hs->u.status_parse.code; - - nxt_debug(task, "upstream version:%d status:%uD \"%*s\"", - hs->u.status_parse.http_version, - hs->u.status_parse.code, - hs->u.status_parse.end - hs->u.status_parse.start, - hs->u.status_parse.start); - - nxt_memzero(&hs->u.header, sizeof(nxt_http_split_header_parse_t)); - hs->u.header.mem_pool = hs->upstream->buffers.mem_pool; - - nxt_http_source_header_filter(task, hs, b); - return; - } - - if (nxt_slow_path(ret == NXT_ERROR)) { - /* HTTP/0.9 response. */ - hs->header_in.status = 200; - nxt_http_source_header_ready(task, hs, b); - return; - } - - /* ret == NXT_AGAIN */ - - /* - * b->mem.pos is always equal to b->mem.end because b is a buffer - * which points to a response part read by the stream source. - * However, since the stream source is an immediate source of the - * status filter, b->parent is a buffer the stream source reads in. - */ - if (b->parent->mem.pos == b->parent->mem.end) { - nxt_http_source_message("upstream sent too long status line: \"%*s\"", - b->mem.pos - b->mem.start, b->mem.start); - - nxt_http_source_fail(task, hs); - } -} - - -static void -nxt_http_source_header_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_int_t ret; - nxt_buf_t *b; - nxt_http_source_t *hs; - - hs = obj; - b = data; - - /* - * No cycle over buffer chain is required since at - * start the stream source passes buffers one at a time. - */ - - nxt_debug(task, "http source header filter"); - - if (nxt_slow_path(nxt_buf_is_sync(b))) { - nxt_http_source_sync_buffer(task, hs, b); - return; - } - - for ( ;; ) { - ret = nxt_http_split_header_parse(&hs->u.header, &b->mem); - - if (nxt_slow_path(ret != NXT_OK)) { - break; - } - - ret = nxt_http_source_header_line_process(hs); - - if (nxt_slow_path(ret != NXT_OK)) { - break; - } - } - - if (nxt_fast_path(ret == NXT_DONE)) { - nxt_debug(task, "http source header done"); - nxt_http_source_header_ready(task, hs, b); - return; - } - - if (nxt_fast_path(ret == NXT_AGAIN)) { - return; - } - - if (ret != NXT_ERROR) { - /* ret == NXT_DECLINED: "\r" is not followed by "\n" */ - nxt_log(task, NXT_LOG_ERR, - "upstream sent invalid header line: \"%*s\\r...\"", - hs->u.header.parse.header_end - - hs->u.header.parse.header_name_start, - hs->u.header.parse.header_name_start); - } - - /* ret == NXT_ERROR */ - - nxt_http_source_fail(task, hs); -} - - -static nxt_int_t -nxt_http_source_header_line_process(nxt_http_source_t *hs) -{ - size_t name_len; - nxt_name_value_t *nv; - nxt_lvlhsh_query_t lhq; - nxt_http_header_parse_t *hp; - nxt_upstream_name_value_t *unv; - - hp = &hs->u.header.parse; - - name_len = hp->header_name_end - hp->header_name_start; - - if (name_len > 255) { - nxt_http_source_message("upstream sent too long header field name: " - "\"%*s\"", name_len, hp->header_name_start); - return NXT_ERROR; - } - - nv = nxt_list_add(hs->header_in.list); - if (nxt_slow_path(nv == NULL)) { - return NXT_ERROR; - } - - nv->hash = hp->header_hash; - nv->skip = 0; - nv->name_len = name_len; - nv->name_start = hp->header_name_start; - nv->value_len = hp->header_end - hp->header_start; - nv->value_start = hp->header_start; - - nxt_thread_log_debug("upstream header: \"%*s: %*s\"", - nv->name_len, nv->name_start, - nv->value_len, nv->value_start); - - lhq.key_hash = nv->hash; - lhq.key.len = nv->name_len; - lhq.key.data = nv->name_start; - lhq.proto = &nxt_upstream_header_hash_proto; - - if (nxt_lvlhsh_find(&hs->header_in.hash, &lhq) == NXT_OK) { - unv = lhq.value; - - if (unv->handler(hs->upstream, nv) != NXT_OK) { - return NXT_ERROR; - } - } - - return NXT_OK; -} - - -static const nxt_upstream_name_value_t nxt_http_source_headers[] - nxt_aligned(32) = -{ - { nxt_http_source_content_length, - nxt_upstream_name_value("content-length") }, - - { nxt_http_source_transfer_encoding, - nxt_upstream_name_value("transfer-encoding") }, -}; - - -nxt_int_t -nxt_http_source_hash_create(nxt_mp_t *mp, nxt_lvlhsh_t *lh) -{ - return nxt_upstream_header_hash_add(mp, lh, nxt_http_source_headers, - nxt_nitems(nxt_http_source_headers)); -} - - -static nxt_int_t -nxt_http_source_content_length(nxt_upstream_source_t *us, nxt_name_value_t *nv) -{ - nxt_off_t length; - nxt_http_source_t *hs; - - length = nxt_off_t_parse(nv->value_start, nv->value_len); - - if (nxt_fast_path(length > 0)) { - hs = us->protocol_source; - hs->header_in.content_length = length; - return NXT_OK; - } - - return NXT_ERROR; -} - - -static nxt_int_t -nxt_http_source_transfer_encoding(nxt_upstream_source_t *us, - nxt_name_value_t *nv) -{ - u_char *end; - nxt_http_source_t *hs; - - end = nv->value_start + nv->value_len; - - if (nxt_memcasestrn(nv->value_start, end, "chunked", 7) != NULL) { - hs = us->protocol_source; - hs->chunked = 1; - } - - return NXT_OK; -} - - -static void -nxt_http_source_header_ready(nxt_task_t *task, nxt_http_source_t *hs, - nxt_buf_t *rest) -{ - nxt_buf_t *b; - nxt_upstream_source_t *us; - nxt_http_source_chunk_t *hsc; - - us = hs->upstream; - - /* Free buffers used for request header. */ - - for (b = us->stream->out; b != NULL; b = b->next) { - nxt_buf_pool_free(&us->buffers, b); - } - - if (nxt_fast_path(nxt_buf_pool_available(&us->buffers))) { - - if (hs->chunked) { - hsc = nxt_mp_zalloc(hs->upstream->buffers.mem_pool, - sizeof(nxt_http_source_chunk_t)); - if (nxt_slow_path(hsc == NULL)) { - goto fail; - } - - /* - * Change the HTTP source filter chain: - * stream source | chunk filter | HTTP body filter - */ - hs->query.context = hsc; - hs->query.filter = nxt_http_source_chunk_filter; - - hsc->next.context = hs; - hsc->next.filter = nxt_http_source_body_filter; - - hsc->parse.mem_pool = hs->upstream->buffers.mem_pool; - - if (nxt_buf_mem_used_size(&rest->mem) != 0) { - hs->rest = nxt_http_chunk_parse(task, &hsc->parse, rest); - - if (nxt_slow_path(hs->rest == NULL)) { - goto fail; - } - } - - } else { - /* - * Change the HTTP source filter chain: - * stream source | HTTP body filter - */ - hs->query.filter = nxt_http_source_body_filter; - - if (nxt_buf_mem_used_size(&rest->mem) != 0) { - hs->rest = rest; - } - } - - hs->upstream->state->ready_handler(hs); - return; - } - - nxt_thread_log_error(NXT_LOG_ERR, "%d buffers %uDK each " - "are not enough to read upstream response", - us->buffers.max, us->buffers.size / 1024); -fail: - - nxt_http_source_fail(task, hs); -} - - -static void -nxt_http_source_chunk_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b; - nxt_http_source_t *hs; - nxt_http_source_chunk_t *hsc; - - hsc = obj; - b = data; - - nxt_debug(task, "http source chunk filter"); - - b = nxt_http_chunk_parse(task, &hsc->parse, b); - - hs = hsc->next.context; - - if (hsc->parse.error) { - nxt_http_source_fail(task, hs); - return; - } - - if (hsc->parse.chunk_error) { - /* Output all parsed before a chunk error and close upstream. */ - nxt_thread_current_work_queue_add(task->thread, - nxt_http_source_chunk_error, - task, hs, NULL); - } - - if (b != NULL) { - nxt_source_filter(task->thread, hs->upstream->work_queue, task, - &hsc->next, b); - } -} - - -static void -nxt_http_source_chunk_error(nxt_task_t *task, void *obj, void *data) -{ - nxt_http_source_t *hs; - - hs = obj; - - nxt_http_source_fail(task, hs); -} - - -/* - * The HTTP source body filter accumulates first body buffers before the next - * filter will be established and sets completion handler for the last buffer. - */ - -static void -nxt_http_source_body_filter(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b, *in; - nxt_http_source_t *hs; - - hs = obj; - in = data; - - nxt_debug(task, "http source body filter"); - - for (b = in; b != NULL; b = b->next) { - - if (nxt_buf_is_last(b)) { - b->data = hs->upstream->data; - b->completion_handler = hs->upstream->state->completion_handler; - } - } - - if (hs->next != NULL) { - nxt_source_filter(task->thread, hs->upstream->work_queue, task, - hs->next, in); - return; - } - - nxt_buf_chain_add(&hs->rest, in); -} - - -static void -nxt_http_source_sync_buffer(nxt_task_t *task, nxt_http_source_t *hs, - nxt_buf_t *b) -{ - if (nxt_buf_is_last(b)) { - nxt_log(task, NXT_LOG_ERR, - "upstream closed prematurely connection"); - - } else { - nxt_log(task, NXT_LOG_ERR,"%ui buffers %uz each are not " - "enough to process upstream response header", - hs->upstream->buffers.max, hs->upstream->buffers.size); - } - - /* The stream source sends only the last and the nobuf sync buffer. */ - - nxt_http_source_fail(task, hs); -} - - -static void -nxt_http_source_error(nxt_task_t *task, nxt_stream_source_t *stream) -{ - nxt_http_source_t *hs; - - nxt_thread_log_debug("http source error"); - - hs = stream->next->context; - nxt_http_source_fail(task, hs); -} - - -static void -nxt_http_source_fail(nxt_task_t *task, nxt_http_source_t *hs) -{ - nxt_debug(task, "http source fail"); - - /* TODO: fail, next upstream, or bad gateway */ - - hs->upstream->state->error_handler(task, hs, NULL); -} - - -static void -nxt_http_source_message(const char *msg, size_t len, u_char *p) -{ - if (len > NXT_MAX_ERROR_STR - 300) { - len = NXT_MAX_ERROR_STR - 300; - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; - } - - nxt_thread_log_error(NXT_LOG_ERR, msg, len, p); -} diff --git a/src/nxt_http_source.h b/src/nxt_http_source.h deleted file mode 100644 index 7cf2876b..00000000 --- a/src/nxt_http_source.h +++ /dev/null @@ -1,47 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_HTTP_SOURCE_H_INCLUDED_ -#define _NXT_HTTP_SOURCE_H_INCLUDED_ - - -typedef struct { - nxt_str_t copy; - uintptr_t data[3]; -} nxt_http_source_request_t; - - -typedef struct nxt_http_source_s nxt_http_source_t; -typedef nxt_int_t (*nxt_http_source_request_create_t)(nxt_http_source_t *hs); - - -struct nxt_http_source_s { - nxt_source_hook_t query; - nxt_source_hook_t *next; - - nxt_upstream_source_t *upstream; - - nxt_http_source_request_create_t request_create; - - nxt_upstream_header_in_t header_in; - - nxt_buf_t *rest; - - uint32_t chunked; /* 1 bit */ - - union { - nxt_http_source_request_t request; - } u; -}; - - -NXT_EXPORT void nxt_http_source_handler(nxt_task_t *task, - nxt_upstream_source_t *us, nxt_http_source_request_create_t request_create); -NXT_EXPORT nxt_int_t nxt_http_source_hash_create(nxt_mp_t *mp, - nxt_lvlhsh_t *lh); - - -#endif /* _NXT_HTTP_SOURCE_H_INCLUDED_ */ diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 68174b9d..5e44aab4 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -196,6 +196,9 @@ nxt_http_static(nxt_task_t *task, nxt_http_request_t *r, if (!nxt_str_eq(r->method, "HEAD", 4)) { if (action->fallback != NULL) { + if (nxt_slow_path(r->log_route)) { + nxt_log(task, NXT_LOG_NOTICE, "\"fallback\" taken"); + } return action->fallback; } @@ -690,6 +693,9 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, } if (action->fallback != NULL) { + if (nxt_slow_path(r->log_route)) { + nxt_log(task, NXT_LOG_NOTICE, "\"fallback\" taken"); + } nxt_http_request_action(task, r, action->fallback); return; } diff --git a/src/nxt_http_variables.c b/src/nxt_http_variables.c index fa0244db..b73d9151 100644 --- a/src/nxt_http_variables.c +++ b/src/nxt_http_variables.c @@ -273,40 +273,11 @@ static nxt_int_t nxt_http_var_request_line(nxt_task_t *task, nxt_str_t *str, void *ctx, uint16_t field) { - size_t length; - u_char *p, *start; nxt_http_request_t *r; r = ctx; - length = r->method->length + 1 + r->target.length + 1 + r->version.length; - - start = nxt_mp_nget(r->mem_pool, length); - if (nxt_slow_path(start == NULL)) { - return NXT_ERROR; - } - - p = start; - - if (r->method->length != 0) { - p = nxt_cpymem(p, r->method->start, r->method->length); - - if (r->target.length != 0) { - *p++ = ' '; - p = nxt_cpymem(p, r->target.start, r->target.length); - - if (r->version.length != 0) { - *p++ = ' '; - p = nxt_cpymem(p, r->version.start, r->version.length); - } - } - - } else { - *p++ = '-'; - } - - str->start = start; - str->length = p - start; + *str = r->request_line; return NXT_OK; } diff --git a/src/nxt_isolation.c b/src/nxt_isolation.c index 614d6bb5..cfa494a8 100644 --- a/src/nxt_isolation.c +++ b/src/nxt_isolation.c @@ -80,6 +80,10 @@ nxt_isolation_main_prefork(nxt_task_t *task, nxt_process_t *process, app_conf = process->data.app; cap_setid = rt->capabilities.setid; +#if (NXT_HAVE_PR_SET_NO_NEW_PRIVS) + process->isolation.new_privs = 1; +#endif + if (app_conf->isolation != NULL) { ret = nxt_isolation_set(task, app_conf->isolation, process); if (nxt_slow_path(ret != NXT_OK)) { diff --git a/src/nxt_job.c b/src/nxt_job.c index 995fd89b..56073953 100644 --- a/src/nxt_job.c +++ b/src/nxt_job.c @@ -32,12 +32,14 @@ nxt_job_create(nxt_mp_t *mp, size_t size) cache_size = size; } - if (nxt_fast_path(job != NULL)) { - job->cache_size = (uint16_t) cache_size; - job->mem_pool = mp; - nxt_job_set_name(job, "job"); + if (nxt_slow_path(job == NULL)) { + return NULL; } + job->cache_size = (uint16_t) cache_size; + job->mem_pool = mp; + nxt_job_set_name(job, "job"); + /* Allow safe nxt_queue_remove() in nxt_job_destroy(). */ nxt_queue_self(&job->link); diff --git a/src/nxt_job_file.c b/src/nxt_job_file.c deleted file mode 100644 index 675bed2f..00000000 --- a/src/nxt_job_file.c +++ /dev/null @@ -1,302 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - - -#include <nxt_main.h> - - -static void nxt_job_file_open_and_read(nxt_task_t *task, void *obj, void *data); -static nxt_int_t nxt_job_file_open(nxt_job_file_t *jbf); -static nxt_int_t nxt_job_file_info(nxt_job_file_t *jbf); -static nxt_int_t nxt_job_file_mmap(nxt_job_file_t *jbf, size_t size); -static nxt_int_t nxt_job_file_read_data(nxt_job_file_t *jbf, size_t size); -static nxt_int_t nxt_job_file_read_required(nxt_job_file_t *jbf); - - -nxt_job_file_t * -nxt_job_file_create(nxt_mp_t *mp) -{ - nxt_job_file_t *jbf; - - jbf = nxt_job_create(mp, sizeof(nxt_job_file_t)); - - if (nxt_fast_path(jbf != NULL)) { - jbf->file.fd = NXT_FILE_INVALID; - jbf->file.accessed = NXT_FILE_ACCESSED_LONG_AGO; - jbf->read_required = nxt_job_file_read_required; - } - - return jbf; -} - - -void -nxt_job_file_init(nxt_job_file_t *jbf) -{ - nxt_job_init(&jbf->job, sizeof(nxt_job_file_t)); - - jbf->file.fd = NXT_FILE_INVALID; - jbf->file.accessed = NXT_FILE_ACCESSED_LONG_AGO; - jbf->read_required = nxt_job_file_read_required; -} - - -/* - * Must be a function but not a macro, because - * it can be used as function pointer. - */ - -void -nxt_job_file_read(nxt_task_t *task, nxt_job_t *job) -{ - nxt_job_start(task, job, nxt_job_file_open_and_read); -} - - -static void -nxt_job_file_open_and_read(nxt_task_t *task, void *obj, void *data) -{ - size_t size; - nxt_int_t n; - nxt_bool_t read_ahead; - nxt_file_t *file; - nxt_job_file_t *jbf; - nxt_work_handler_t handler; - - jbf = obj; - file = &jbf->file; - - nxt_debug(task, "file job read: \"%FN\"", file->name); - - if (file->fd != NXT_FILE_INVALID && jbf->close_before_open) { - nxt_file_close(file); - file->fd = NXT_FILE_INVALID; - } - - if (file->fd == NXT_FILE_INVALID) { - - switch (nxt_job_file_open(jbf)) { - - case NXT_OK: - break; - - case NXT_DECLINED: - handler = jbf->ready_handler; - goto done; - - default: /* NXT_ERROR */ - handler = jbf->error_handler; - goto done; - } - } - - if (file->size > 0) { - - if (jbf->buffer != NULL) { - size = nxt_buf_mem_size(&jbf->buffer->mem); - size = nxt_min(file->size, (nxt_off_t) size); - read_ahead = nxt_buf_is_mmap(jbf->buffer); - - } else { - size = nxt_min(file->size, 1024 * 1024); - read_ahead = jbf->read_ahead; - } - - if (read_ahead) { - nxt_file_read_ahead(&jbf->file, jbf->offset, size); - } - - if (jbf->buffer != NULL) { - - if (nxt_buf_is_mmap(jbf->buffer)) { - n = nxt_job_file_mmap(jbf, size); - - } else { - n = nxt_job_file_read_data(jbf, size); - } - - if (nxt_slow_path(n != NXT_OK)) { - handler = jbf->error_handler; - goto done; - } - } - } - - if (jbf->offset == file->size) { - jbf->complete = 1; - - if (jbf->close) { - nxt_file_close(file); - file->fd = NXT_FILE_INVALID; - } - } - - nxt_job_return(task, &jbf->job, jbf->ready_handler); - return; - -done: - - if (file->fd != NXT_FILE_INVALID) { - nxt_file_close(file); - file->fd = NXT_FILE_INVALID; - } - - nxt_job_return(task, &jbf->job, handler); -} - - -static nxt_int_t -nxt_job_file_open(nxt_job_file_t *jbf) -{ - nxt_int_t n; - - if (jbf->test_before_open) { - n = nxt_job_file_info(jbf); - - if (n != NXT_OK) { - goto test_directory; - } - - if (jbf->file.type == NXT_FILE_DIRECTORY) { - return NXT_DECLINED; - } - - if (jbf->read_required(jbf) != NXT_OK) { - return NXT_DECLINED; - } - } - - n = nxt_file_open(&jbf->file, NXT_FILE_RDONLY, NXT_FILE_OPEN, 0); - - if (n == NXT_OK) { - n = nxt_job_file_info(jbf); - - if (nxt_fast_path(n == NXT_OK)) { - - if (jbf->file.type == NXT_FILE_DIRECTORY) { - return NXT_DECLINED; - } - - return jbf->read_required(jbf); - } - - return n; - } - -test_directory: - - if (jbf->directory_end != 0 - && jbf->file.error != NXT_ENOTDIR - && jbf->file.error != NXT_ENAMETOOLONG - && jbf->file.error != NXT_EACCES) - { - jbf->file.name[jbf->directory_end] = '\0'; - - return nxt_job_file_info(jbf); - } - - return n; -} - - -static nxt_int_t -nxt_job_file_info(nxt_job_file_t *jbf) -{ - nxt_int_t n; - nxt_file_t *file; - nxt_file_info_t fi; - - file = &jbf->file; - - n = nxt_file_info(file, &fi); - - if (n != NXT_OK) { - return NXT_ERROR; - } - - if (nxt_is_file(&fi)) { - file->type = NXT_FILE_REGULAR; - file->size = nxt_file_size(&fi); - file->mtime = nxt_file_mtime(&fi); - - } else if (nxt_is_dir(&fi)) { - file->type = NXT_FILE_DIRECTORY; - file->size = nxt_file_size(&fi); - file->mtime = nxt_file_mtime(&fi); - } - - return NXT_OK; -} - - -static nxt_int_t -nxt_job_file_mmap(nxt_job_file_t *jbf, size_t size) -{ - u_char *p, *end; - static nxt_uint_t n; - - p = nxt_mem_map(NULL, &jbf->buffer->mmap, size, NXT_MEM_MAP_READ, - (NXT_MEM_MAP_FILE | NXT_MEM_MAP_PREFAULT), - jbf->file.fd, jbf->offset); - - if (nxt_fast_path(p != NXT_MEM_MAP_FAILED)) { - - end = p + size; - - jbf->buffer->mem.pos = p; - jbf->buffer->mem.free = end; - jbf->buffer->mem.start = p; - jbf->buffer->mem.end = end; - jbf->buffer->file_end += size; - jbf->offset += size; - - /* - * The mapped pages should be already preloaded in the kernel page - * cache by nxt_file_read_ahead(). Touching them should wire the pages - * in user land memory if mmap() did not do this. Adding to the static - * variable "n" disables the loop elimination during optimization. - */ - n += *p; - - for (p = nxt_align_ptr(p, nxt_pagesize); p < end; p += nxt_pagesize) { - n += *p; - } - - return NXT_OK; - } - - return NXT_ERROR; -} - - -static nxt_int_t -nxt_job_file_read_data(nxt_job_file_t *jbf, size_t size) -{ - ssize_t n; - - n = nxt_file_read(&jbf->file, jbf->buffer->mem.free, size, jbf->offset); - - if (nxt_fast_path(n > 0)) { - - jbf->buffer->mem.free += n; - jbf->offset += n; - - if (nxt_buf_is_file(jbf->buffer)) { - jbf->buffer->file_end += n; - } - - return NXT_OK; - } - - return NXT_ERROR; -} - - -static nxt_int_t -nxt_job_file_read_required(nxt_job_file_t *jbf) -{ - return NXT_OK; -} diff --git a/src/nxt_job_file.h b/src/nxt_job_file.h deleted file mode 100644 index 93c6393c..00000000 --- a/src/nxt_job_file.h +++ /dev/null @@ -1,74 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_JOB_FILE_H_INCLUDED_ -#define _NXT_JOB_FILE_H_INCLUDED_ - - -/* - * nxt_job_file_read() allows to open a file, to get its type, size, and - * modification time, to read or map file content to memory, and to close - * the file. It can be done as one operation for small file or as several - * operations for large file. On each operation completion ready_handler - * or error_handler completion handlers are called. Since they are job - * operations, they can be run by a thread pool. - * - * If a file is not opened then it is opened and its type, size, and - * modification time are got. Then file content starting from given offset - * is read or mapped in memory if there is a buffer supplied. The offset - * field is correspondingly updated. - * - * If there is no buffer but the read_ahead flag is set then the first - * byte is read to initiate read ahead operation. - * - * If the close flag is set then file descriptor is closed when the file - * is completely read. - * - * The complete flag is set by nxt_job_file_read() when the file is - * completely read. - * - * The test_before_open flag allows to save syscalls in some case, for - * example, not to open and then not to close a directory. It calls - * nxt_file_info() to get file type, size, and modification time before - * opening the file. A custom read_required() callback combined with this - * flag can also omit opening and reading on some conditions. However, - * if the callback forces opening then additional nxt_file_info() is - * called after opening. The default read_required() callback always - * forces opening and reading. - */ - - -typedef struct nxt_job_file_s nxt_job_file_t; - -struct nxt_job_file_s { - nxt_job_t job; - - nxt_file_t file; - - nxt_off_t offset; - nxt_buf_t *buffer; - - nxt_work_handler_t ready_handler; - nxt_work_handler_t error_handler; - - nxt_int_t (*read_required)(nxt_job_file_t *jbf); - - uint16_t directory_end; - - uint16_t close_before_open:1; - uint16_t test_before_open:1; - uint16_t read_ahead:1; - uint16_t close:1; - uint16_t complete:1; -}; - - -NXT_EXPORT nxt_job_file_t *nxt_job_file_create(nxt_mp_t *mp); -NXT_EXPORT void nxt_job_file_init(nxt_job_file_t *jbf); -NXT_EXPORT void nxt_job_file_read(nxt_task_t *task, nxt_job_t *job); - - -#endif /* _NXT_JOB_FILE_H_INCLUDED_ */ diff --git a/src/nxt_job_file_cache.c b/src/nxt_job_file_cache.c deleted file mode 100644 index 680d0665..00000000 --- a/src/nxt_job_file_cache.c +++ /dev/null @@ -1,42 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - - -#include <nxt_main.h> - - -typedef struct { - nxt_cache_node_t node; - nxt_file_t file; -} nxt_file_cache_t; - - -void -nxt_job_file_cache_read(nxt_cache_t *cache, nxt_job_file_t *jbf) -{ - nxt_file_cache_node_t *node; - - node = nxt_cache_find(cache); - - if (node != NULL) { - - if (node->fd != -1) { - nxt_job_return(&jbf->job, jbf->ready_handler); - return; - } - - if (node->error != 0) { - nxt_job_return(&jbf->job, jbf->error_handler); - return; - } - - if (node->accessed + 60 > nxt_thread_time()) { - jbf->job.thread_pool = NULL; - } - } - - nxt_job_file_read(jbf); -} diff --git a/src/nxt_js.c b/src/nxt_js.c index 4327e848..df945db6 100644 --- a/src/nxt_js.c +++ b/src/nxt_js.c @@ -8,16 +8,72 @@ struct nxt_js_s { uint32_t index; - njs_vm_t *vm; }; +typedef struct { + nxt_str_t name; + nxt_str_t text; +} nxt_js_module_t; + + struct nxt_js_conf_s { nxt_mp_t *pool; njs_vm_t *vm; njs_uint_t protos; njs_external_t *proto; + nxt_str_t init; + nxt_array_t *modules; /* of nxt_js_module_t */ nxt_array_t *funcs; + uint8_t test; /* 1 bit */ +}; + + +njs_mod_t * +nxt_js_module_loader(njs_vm_t *vm, njs_external_ptr_t external, njs_str_t *name) +{ + nxt_str_t text; + nxt_uint_t i, n; + nxt_js_conf_t *jcf; + nxt_js_module_t *modules, *module; + + jcf = external; + + module = NULL; + + n = jcf->modules->nelts; + modules = jcf->modules->elts; + + for (i = 0; i < n; i++) { + if (nxt_strstr_eq(name, &modules[i].name)) { + module = &modules[i]; + break; + } + } + + if (module == NULL) { + return NULL; + } + + text.length = module->text.length; + + text.start = njs_mp_alloc(vm->mem_pool, text.length); + if (nxt_slow_path(text.start == NULL)) { + return NULL; + } + + nxt_memcpy(text.start, module->text.start, text.length); + + return njs_vm_compile_module(vm, name, &text.start, + &text.start[text.length]); +} + + +static njs_vm_ops_t nxt_js_ops = { + NULL, + NULL, + nxt_js_module_loader, + NULL, }; @@ -25,9 +81,8 @@ njs_int_t nxt_js_proto_id; nxt_js_conf_t * -nxt_js_conf_new(nxt_mp_t *mp) +nxt_js_conf_new(nxt_mp_t *mp, nxt_bool_t test) { - njs_vm_opt_t opts; nxt_js_conf_t *jcf; jcf = nxt_mp_zget(mp, sizeof(nxt_js_conf_t)); @@ -36,17 +91,15 @@ nxt_js_conf_new(nxt_mp_t *mp) } jcf->pool = mp; + jcf->test = test; - njs_vm_opt_init(&opts); - - jcf->vm = njs_vm_create(&opts); - if (nxt_slow_path(jcf->vm == NULL)) { + jcf->modules = nxt_array_create(mp, 4, sizeof(nxt_js_module_t)); + if (nxt_slow_path(jcf->modules == NULL)) { return NULL; } jcf->funcs = nxt_array_create(mp, 4, sizeof(nxt_str_t)); if (nxt_slow_path(jcf->funcs == NULL)) { - njs_vm_destroy(jcf->vm); return NULL; } @@ -69,6 +122,115 @@ nxt_js_set_proto(nxt_js_conf_t *jcf, njs_external_t *proto, njs_uint_t n) } +static njs_vm_t * +nxt_js_vm_create(nxt_js_conf_t *jcf) +{ + u_char *p; + size_t size; + nxt_uint_t i; + njs_vm_opt_t opts; + nxt_js_module_t *module, *mod; + + static nxt_str_t import_str = nxt_string("import"); + static nxt_str_t from_str = nxt_string("from"); + static nxt_str_t global_str = nxt_string("globalThis"); + + njs_vm_opt_init(&opts); + + opts.backtrace = 1; + + opts.file.start = (u_char *) "default"; + opts.file.length = 7; + + if (jcf->test || jcf->modules->nelts == 0) { + goto done; + } + + opts.ops = &nxt_js_ops; + opts.external = jcf; + + size = 0; + module = jcf->modules->elts; + + for (i = 0; i < jcf->modules->nelts; i++) { + mod = &module[i]; + + size += import_str.length + 1 + mod->name.length + 1 + + from_str.length + 2 + mod->name.length + 3; + + size += global_str.length + 1 + mod->name.length + 3 + + mod->name.length + 2; + } + + p = nxt_mp_nget(jcf->pool, size); + if (nxt_slow_path(p == NULL)) { + return NULL; + } + + jcf->init.length = size; + jcf->init.start = p; + + for (i = 0; i < jcf->modules->nelts; i++) { + mod = &module[i]; + + p = nxt_cpymem(p, import_str.start, import_str.length); + *p++ = ' '; + + p = nxt_cpymem(p, mod->name.start, mod->name.length); + *p++ = ' '; + + p = nxt_cpymem(p, from_str.start, from_str.length); + *p++ = ' '; + + *p++ = '\"'; + p = nxt_cpymem(p, mod->name.start, mod->name.length); + *p++ = '\"'; + *p++ = ';'; + *p++ = '\n'; + + p = nxt_cpymem(p, global_str.start, global_str.length); + *p++ = '.'; + + p = nxt_cpymem(p, mod->name.start, mod->name.length); + *p++ = ' '; + *p++ = '='; + *p++ = ' '; + + p = nxt_cpymem(p, mod->name.start, mod->name.length); + *p++ = ';'; + *p++ = '\n'; + } + +done: + + return njs_vm_create(&opts); +} + + +nxt_int_t +nxt_js_add_module(nxt_js_conf_t *jcf, nxt_str_t *name, nxt_str_t *text) +{ + nxt_js_module_t *module; + + module = nxt_array_add(jcf->modules); + if (nxt_slow_path(module == NULL)) { + return NXT_ERROR; + } + + module->name = *name; + + module->text.length = text->length; + module->text.start = nxt_mp_nget(jcf->pool, text->length); + if (nxt_slow_path(module->text.start == NULL)) { + return NXT_ERROR; + } + + nxt_memcpy(module->text.start, text->start, text->length); + + return NXT_OK; +} + + nxt_js_t * nxt_js_add_tpl(nxt_js_conf_t *jcf, nxt_str_t *str, nxt_bool_t strz) { @@ -113,8 +275,6 @@ nxt_js_add_tpl(nxt_js_conf_t *jcf, nxt_str_t *str, nxt_bool_t strz) return NULL; } - js->vm = jcf->vm; - func = nxt_array_add(jcf->funcs); if (nxt_slow_path(func == NULL)) { return NULL; @@ -138,7 +298,16 @@ nxt_js_compile(nxt_js_conf_t *jcf) nxt_str_t *func; nxt_uint_t i; - size = 2; + if (jcf->test) { + return NXT_OK; + } + + jcf->vm = nxt_js_vm_create(jcf); + if (nxt_slow_path(jcf->vm == NULL)) { + return NXT_ERROR; + } + + size = jcf->init.length + 2; func = jcf->funcs->elts; for (i = 0; i < jcf->funcs->nelts; i++) { @@ -150,7 +319,7 @@ nxt_js_compile(nxt_js_conf_t *jcf) return NXT_ERROR; } - p = start; + p = nxt_cpymem(start, jcf->init.start, jcf->init.length); *p++ = '['; func = jcf->funcs->elts; @@ -178,37 +347,43 @@ nxt_int_t nxt_js_test(nxt_js_conf_t *jcf, nxt_str_t *str, u_char *error) { u_char *start; - nxt_str_t err; + njs_vm_t *vm; njs_int_t ret; - njs_str_t res; + + vm = nxt_js_vm_create(jcf); + if (nxt_slow_path(vm == NULL)) { + return NXT_ERROR; + } start = nxt_mp_nget(jcf->pool, str->length); if (nxt_slow_path(start == NULL)) { - return NXT_ERROR; + goto fail; } nxt_memcpy(start, str->start, str->length); - ret = njs_vm_compile(jcf->vm, &start, start + str->length); + ret = njs_vm_compile(vm, &start, start + str->length); if (nxt_slow_path(ret != NJS_OK)) { - (void) njs_vm_retval_string(jcf->vm, &res); + (void) nxt_js_error(vm, error); + goto fail; + } + + njs_vm_destroy(vm); - err.start = res.start; - err.length = res.length; + return NXT_OK; - nxt_sprintf(error, error + NXT_MAX_ERROR_STR, "\"%V\"%Z", &err); +fail: - return NXT_ERROR; - } + njs_vm_destroy(vm); - return NXT_OK; + return NXT_ERROR; } nxt_int_t -nxt_js_call(nxt_task_t *task, nxt_js_cache_t *cache, nxt_js_t *js, - nxt_str_t *str, void *ctx) +nxt_js_call(nxt_task_t *task, nxt_js_conf_t *jcf, nxt_js_cache_t *cache, + nxt_js_t *js, nxt_str_t *str, void *ctx) { njs_vm_t *vm; njs_int_t rc, ret; @@ -227,7 +402,7 @@ nxt_js_call(nxt_task_t *task, nxt_js_cache_t *cache, nxt_js_t *js, vm = cache->vm; if (vm == NULL) { - vm = njs_vm_clone(js->vm, ctx); + vm = njs_vm_clone(jcf->vm, ctx); if (nxt_slow_path(vm == NULL)) { return NXT_ERROR; } @@ -314,3 +489,24 @@ nxt_js_release(nxt_js_cache_t *cache) njs_vm_destroy(cache->vm); } } + + +nxt_int_t +nxt_js_error(njs_vm_t *vm, u_char *error) +{ + njs_int_t ret; + njs_str_t res; + nxt_str_t err; + + ret = njs_vm_retval_string(vm, &res); + if (nxt_slow_path(ret != NJS_OK)) { + return NXT_ERROR; + } + + err.start = res.start; + err.length = res.length; + + nxt_sprintf(error, error + NXT_MAX_ERROR_STR, "\"%V\"%Z", &err); + + return NXT_OK; +} diff --git a/src/nxt_js.h b/src/nxt_js.h index 74d041ca..48f036b8 100644 --- a/src/nxt_js.h +++ b/src/nxt_js.h @@ -21,15 +21,20 @@ typedef struct { } nxt_js_cache_t; -nxt_js_conf_t *nxt_js_conf_new(nxt_mp_t *mp); +njs_mod_t *nxt_js_module_loader(njs_vm_t *vm, njs_external_ptr_t external, + njs_str_t *name); +nxt_js_conf_t *nxt_js_conf_new(nxt_mp_t *mp, nxt_bool_t test); void nxt_js_conf_release(nxt_js_conf_t *jcf); void nxt_js_set_proto(nxt_js_conf_t *jcf, njs_external_t *proto, nxt_uint_t n); +nxt_int_t nxt_js_add_module(nxt_js_conf_t *jcf, nxt_str_t *name, + nxt_str_t *text); nxt_js_t *nxt_js_add_tpl(nxt_js_conf_t *jcf, nxt_str_t *str, nxt_bool_t strz); nxt_int_t nxt_js_compile(nxt_js_conf_t *jcf); nxt_int_t nxt_js_test(nxt_js_conf_t *jcf, nxt_str_t *str, u_char *error); -nxt_int_t nxt_js_call(nxt_task_t *task, nxt_js_cache_t *cache, nxt_js_t *js, - nxt_str_t *str, void *ctx); +nxt_int_t nxt_js_call(nxt_task_t *task, nxt_js_conf_t *jcf, + nxt_js_cache_t *cache, nxt_js_t *js, nxt_str_t *str, void *ctx); void nxt_js_release(nxt_js_cache_t *cache); +nxt_int_t nxt_js_error(njs_vm_t *vm, u_char *error); extern njs_int_t nxt_js_proto_id; diff --git a/src/nxt_kqueue_engine.c b/src/nxt_kqueue_engine.c index ecc3251e..a7a5a29e 100644 --- a/src/nxt_kqueue_engine.c +++ b/src/nxt_kqueue_engine.c @@ -716,6 +716,8 @@ nxt_kqueue_poll(nxt_event_engine_t *engine, nxt_msec_t timeout) for (i = 0; i < nevents; i++) { + error = 0; + kev = &engine->u.kqueue.events[i]; nxt_debug(&engine->task, @@ -725,12 +727,11 @@ nxt_kqueue_poll(nxt_event_engine_t *engine, nxt_msec_t timeout) kev->ident, kev->filter, kev->flags, kev->fflags, kev->data, kev->udata); - error = (kev->flags & EV_ERROR); - - if (nxt_slow_path(error)) { + if (nxt_slow_path(kev->flags & EV_ERROR)) { nxt_alert(&engine->task, "kevent(%d) error %E on ident:%d filter:%d", engine->u.kqueue.fd, kev->data, kev->ident, kev->filter); + error = 1; } task = &engine->task; diff --git a/src/nxt_listen_socket.c b/src/nxt_listen_socket.c index f10abdef..d477eef1 100644 --- a/src/nxt_listen_socket.c +++ b/src/nxt_listen_socket.c @@ -161,7 +161,7 @@ nxt_listen_socket_create(nxt_task_t *task, nxt_mp_t *mp, nxt_socket_close(task, ts); if (ret == 0) { - nxt_alert(task, "connect(%d, %*s) succeed, address already in use", + nxt_alert(task, "connect(%d, %*s) socket already in use", ts, (size_t) orig_sa->length, nxt_sockaddr_start(orig_sa)); diff --git a/src/nxt_main.h b/src/nxt_main.h index b0cdc2d3..a7e0c283 100644 --- a/src/nxt_main.h +++ b/src/nxt_main.h @@ -11,7 +11,8 @@ #include <nxt_auto_config.h> #include <nxt_version.h> -#define NXT_SERVER "Unit/" NXT_VERSION +#define NXT_NAME "Unit" +#define NXT_SERVER NXT_NAME "/" NXT_VERSION typedef struct nxt_port_s nxt_port_t; typedef struct nxt_task_s nxt_task_t; @@ -145,13 +146,9 @@ typedef void (*nxt_event_conn_handler_t)(nxt_thread_t *thr, nxt_conn_t *c); #include <nxt_event_engine.h> #include <nxt_job.h> -#include <nxt_job_file.h> -#include <nxt_buf_filter.h> #include <nxt_sockaddr.h> -#include <nxt_cache.h> - #include <nxt_http_parse.h> #include <nxt_runtime.h> #include <nxt_port_hash.h> diff --git a/src/nxt_main_process.c b/src/nxt_main_process.c index 4c89121e..7cba08d4 100644 --- a/src/nxt_main_process.c +++ b/src/nxt_main_process.c @@ -14,6 +14,9 @@ #if (NXT_TLS) #include <nxt_cert.h> #endif +#if (NXT_HAVE_NJS) +#include <nxt_script.h> +#endif #include <sys/mount.h> @@ -48,6 +51,8 @@ static void nxt_main_process_signal_handler(nxt_task_t *task, void *obj, static void nxt_main_process_cleanup(nxt_task_t *task, nxt_process_t *process); static void nxt_main_port_socket_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg); +static void nxt_main_port_socket_unlink_handler(nxt_task_t *task, + nxt_port_recv_msg_t *msg); static nxt_int_t nxt_main_listening_socket(nxt_sockaddr_t *sa, nxt_listening_socket_t *ls); static void nxt_main_port_modules_handler(nxt_task_t *task, @@ -119,6 +124,18 @@ static nxt_conf_map_t nxt_common_app_conf[] = { }, { + nxt_string("stdout"), + NXT_CONF_MAP_CSTRZ, + offsetof(nxt_common_app_conf_t, stdout_log), + }, + + { + nxt_string("stderr"), + NXT_CONF_MAP_CSTRZ, + offsetof(nxt_common_app_conf_t, stderr_log), + }, + + { nxt_string("working_directory"), NXT_CONF_MAP_CSTRZ, offsetof(nxt_common_app_conf_t, working_directory), @@ -587,12 +604,17 @@ static nxt_port_handlers_t nxt_main_process_port_handlers = { .remove_pid = nxt_port_remove_pid_handler, .start_process = nxt_main_start_process_handler, .socket = nxt_main_port_socket_handler, + .socket_unlink = nxt_main_port_socket_unlink_handler, .modules = nxt_main_port_modules_handler, .conf_store = nxt_main_port_conf_store_handler, #if (NXT_TLS) .cert_get = nxt_cert_store_get_handler, .cert_delete = nxt_cert_store_delete_handler, #endif +#if (NXT_HAVE_NJS) + .script_get = nxt_script_store_get_handler, + .script_delete = nxt_script_store_delete_handler, +#endif .access_log = nxt_main_port_access_log_handler, .rpc_ready = nxt_port_rpc_handler, .rpc_error = nxt_port_rpc_handler, @@ -1182,8 +1204,9 @@ nxt_main_listening_socket(nxt_sockaddr_t *sa, nxt_listening_socket_t *ls) if (sa->u.sockaddr.sa_family == AF_UNIX && sa->u.sockaddr_un.sun_path[0] != '\0') { - char *filename; - mode_t access; + char *filename; + mode_t access; + nxt_thread_t *thr; filename = sa->u.sockaddr_un.sun_path; access = (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH); @@ -1194,6 +1217,9 @@ nxt_main_listening_socket(nxt_sockaddr_t *sa, nxt_listening_socket_t *ls) filename, nxt_errno); goto fail; } + + thr = nxt_thread(); + nxt_runtime_listen_socket_add(thr->runtime, sa); } #endif @@ -1210,6 +1236,49 @@ fail: } +static void +nxt_main_port_socket_unlink_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg) +{ +#if (NXT_HAVE_UNIX_DOMAIN) + size_t i; + nxt_buf_t *b; + const char *filename; + nxt_runtime_t *rt; + nxt_sockaddr_t *sa; + nxt_listen_socket_t *ls; + + b = msg->buf; + sa = (nxt_sockaddr_t *) b->mem.pos; + + filename = sa->u.sockaddr_un.sun_path; + unlink(filename); + + rt = task->thread->runtime; + + for (i = 0; i < rt->listen_sockets->nelts; i++) { + const char *name; + + ls = (nxt_listen_socket_t *) rt->listen_sockets->elts + i; + sa = ls->sockaddr; + + if (sa->u.sockaddr.sa_family != AF_UNIX + || sa->u.sockaddr_un.sun_path[0] == '\0') + { + continue; + } + + name = sa->u.sockaddr_un.sun_path; + if (strcmp(name, filename) != 0) { + continue; + } + + nxt_array_remove(rt->listen_sockets, ls); + break; + } +#endif +} + + static nxt_conf_map_t nxt_app_lang_module_map[] = { { nxt_string("type"), diff --git a/src/nxt_mem_pool_cleanup.c b/src/nxt_mem_pool_cleanup.c deleted file mode 100644 index ceafc9c8..00000000 --- a/src/nxt_mem_pool_cleanup.c +++ /dev/null @@ -1,39 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -static void nxt_mem_pool_file_cleanup_handler(nxt_task_t *task, void *data); - - -nxt_mem_pool_cleanup_t * -nxt_mem_pool_file_cleanup(nxt_mem_pool_t *mp, nxt_file_t *file) -{ - nxt_mem_pool_cleanup_t *mpcl; - - mpcl = nxt_mem_pool_cleanup(mp, 0); - - if (nxt_fast_path(mpcl != NULL)) { - mpcl->handler = nxt_mem_pool_file_cleanup_handler; - mpcl->data = file; - } - - return mpcl; -} - - -static void -nxt_mem_pool_file_cleanup_handler(nxt_task_t *task, void *data) -{ - nxt_file_t *file; - - file = data; - - if (file->fd != NXT_FILE_INVALID) { - nxt_file_close(task, file); - } -} diff --git a/src/nxt_mem_pool_cleanup.h b/src/nxt_mem_pool_cleanup.h deleted file mode 100644 index f84395d0..00000000 --- a/src/nxt_mem_pool_cleanup.h +++ /dev/null @@ -1,15 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_MEM_POOL_CLEANUP_H_INCLUDED_ -#define _NXT_MEM_POOL_CLEANUP_H_INCLUDED_ - - -NXT_EXPORT nxt_mem_pool_cleanup_t *nxt_mem_pool_file_cleanup(nxt_mem_pool_t *mp, - nxt_file_t *file); - - -#endif /* _NXT_MEM_POOL_CLEANUP_H_INCLUDED_ */ diff --git a/src/nxt_mem_zone.c b/src/nxt_mem_zone.c index f8ab09d9..a3ba3700 100644 --- a/src/nxt_mem_zone.c +++ b/src/nxt_mem_zone.c @@ -672,7 +672,7 @@ nxt_mem_zone_alloc_pages(nxt_mem_zone_t *zone, size_t alignment, uint32_t pages) prev_size = p - (u_char *) block; if (prev_size != 0) { - prev_pages = prev_size >>= zone->page_size_shift; + prev_pages = prev_size >> zone->page_size_shift; node_pages -= prev_pages; block->size = prev_pages; diff --git a/src/nxt_php_sapi.c b/src/nxt_php_sapi.c index d2494938..ba000fc0 100644 --- a/src/nxt_php_sapi.c +++ b/src/nxt_php_sapi.c @@ -102,10 +102,15 @@ static void nxt_php_str_trim_lead(nxt_str_t *str, u_char t); nxt_inline u_char *nxt_realpath(const void *c); static nxt_int_t nxt_php_do_301(nxt_unit_request_info_t *req); +static nxt_int_t nxt_php_handle_fs_err(nxt_unit_request_info_t *req); static void nxt_php_request_handler(nxt_unit_request_info_t *req); static void nxt_php_dynamic_request(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r); +#if (PHP_VERSION_ID < 70400) +static void nxt_zend_stream_init_fp(zend_file_handle *handle, FILE *fp, + const char *filename); +#endif static void nxt_php_execute(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r); nxt_inline void nxt_php_vcwd_chdir(nxt_unit_request_info_t *req, u_char *dir); @@ -980,6 +985,24 @@ nxt_php_do_301(nxt_unit_request_info_t *req) } +static nxt_int_t +nxt_php_handle_fs_err(nxt_unit_request_info_t *req) +{ + switch (nxt_errno) { + case ELOOP: + case EACCES: + case ENFILE: + return nxt_unit_response_init(req, NXT_HTTP_FORBIDDEN, 0, 0); + case ENOENT: + case ENOTDIR: + case ENAMETOOLONG: + return nxt_unit_response_init(req, NXT_HTTP_NOT_FOUND, 0, 0); + } + + return NXT_UNIT_ERROR; +} + + static void nxt_php_request_handler(nxt_unit_request_info_t *req) { @@ -1058,6 +1081,8 @@ nxt_php_dynamic_request(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r) ret = stat(tpath, &sb); if (ret == 0 && S_ISDIR(sb.st_mode)) { ec = nxt_php_do_301(ctx->req); + } else if (ret == -1) { + ec = nxt_php_handle_fs_err(ctx->req); } nxt_unit_request_done(ctx->req, ec); @@ -1109,17 +1134,46 @@ nxt_php_dynamic_request(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r) } +#if (PHP_VERSION_ID < 70400) +static void +nxt_zend_stream_init_fp(zend_file_handle *handle, FILE *fp, + const char *filename) +{ + nxt_memzero(handle, sizeof(zend_file_handle)); + handle->type = ZEND_HANDLE_FP; + handle->handle.fp = fp; + handle->filename = filename; +} +#else +#define nxt_zend_stream_init_fp zend_stream_init_fp +#endif + + static void nxt_php_execute(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r) { + FILE *fp; #if (PHP_VERSION_ID < 50600) void *read_post; #endif + const char *filename; nxt_unit_field_t *f; zend_file_handle file_handle; - nxt_unit_req_debug(ctx->req, "PHP execute script %s", - ctx->script_filename.start); + filename = (const char *) ctx->script_filename.start; + + nxt_unit_req_debug(ctx->req, "PHP execute script %s", filename); + + fp = fopen(filename, "re"); + if (fp == NULL) { + nxt_int_t ec; + + nxt_unit_req_debug(ctx->req, "PHP fopen(\"%s\") failed", filename); + + ec = nxt_php_handle_fs_err(ctx->req); + nxt_unit_request_done(ctx->req, ec); + return; + } SG(server_context) = ctx; SG(options) |= SAPI_OPTION_NO_CHDIR; @@ -1179,16 +1233,7 @@ nxt_php_execute(nxt_php_run_ctx_t *ctx, nxt_unit_request_t *r) nxt_php_vcwd_chdir(ctx->req, ctx->script_dirname.start); } - nxt_memzero(&file_handle, sizeof(file_handle)); - - file_handle.type = ZEND_HANDLE_FILENAME; -#if (PHP_VERSION_ID >= 80100) - file_handle.filename = zend_string_init((char *) ctx->script_filename.start, - ctx->script_filename.length, 0); - file_handle.primary_script = 1; -#else - file_handle.filename = (char *) ctx->script_filename.start; -#endif + nxt_zend_stream_init_fp(&file_handle, fp, filename); php_execute_script(&file_handle TSRMLS_CC); @@ -1487,14 +1532,23 @@ static void nxt_php_set_sptr(nxt_unit_request_info_t *req, const char *name, nxt_unit_sptr_t *v, uint32_t len, zval *track_vars_array TSRMLS_DC) { - char *str; + char *str; +#if NXT_PHP7 + size_t new_len; +#else + unsigned int new_len; +#endif str = nxt_unit_sptr_get(v); nxt_unit_req_debug(req, "php: register %s='%.*s'", name, (int) len, str); - php_register_variable_safe((char *) name, str, len, - track_vars_array TSRMLS_CC); + if (sapi_module.input_filter(PARSE_SERVER, (char *) name, &str, len, + &new_len TSRMLS_CC)) + { + php_register_variable_safe((char *) name, str, new_len, + track_vars_array TSRMLS_CC); + } } diff --git a/src/nxt_port.h b/src/nxt_port.h index 3a8da5ad..772fb41a 100644 --- a/src/nxt_port.h +++ b/src/nxt_port.h @@ -16,10 +16,13 @@ struct nxt_port_handlers_s { /* Main process RPC requests. */ nxt_port_handler_t start_process; nxt_port_handler_t socket; + nxt_port_handler_t socket_unlink; nxt_port_handler_t modules; nxt_port_handler_t conf_store; nxt_port_handler_t cert_get; nxt_port_handler_t cert_delete; + nxt_port_handler_t script_get; + nxt_port_handler_t script_delete; nxt_port_handler_t access_log; /* File descriptor exchange. */ @@ -81,10 +84,13 @@ typedef enum { _NXT_PORT_MSG_START_PROCESS = nxt_port_handler_idx(start_process), _NXT_PORT_MSG_SOCKET = nxt_port_handler_idx(socket), + _NXT_PORT_MSG_SOCKET_UNLINK = nxt_port_handler_idx(socket_unlink), _NXT_PORT_MSG_MODULES = nxt_port_handler_idx(modules), _NXT_PORT_MSG_CONF_STORE = nxt_port_handler_idx(conf_store), _NXT_PORT_MSG_CERT_GET = nxt_port_handler_idx(cert_get), _NXT_PORT_MSG_CERT_DELETE = nxt_port_handler_idx(cert_delete), + _NXT_PORT_MSG_SCRIPT_GET = nxt_port_handler_idx(script_get), + _NXT_PORT_MSG_SCRIPT_DELETE = nxt_port_handler_idx(script_delete), _NXT_PORT_MSG_ACCESS_LOG = nxt_port_handler_idx(access_log), _NXT_PORT_MSG_CHANGE_FILE = nxt_port_handler_idx(change_file), @@ -122,10 +128,13 @@ typedef enum { NXT_PORT_MSG_RPC_ERROR = nxt_msg_last(_NXT_PORT_MSG_RPC_ERROR), NXT_PORT_MSG_START_PROCESS = nxt_msg_last(_NXT_PORT_MSG_START_PROCESS), NXT_PORT_MSG_SOCKET = nxt_msg_last(_NXT_PORT_MSG_SOCKET), + NXT_PORT_MSG_SOCKET_UNLINK = nxt_msg_last(_NXT_PORT_MSG_SOCKET_UNLINK), NXT_PORT_MSG_MODULES = nxt_msg_last(_NXT_PORT_MSG_MODULES), NXT_PORT_MSG_CONF_STORE = nxt_msg_last(_NXT_PORT_MSG_CONF_STORE), NXT_PORT_MSG_CERT_GET = nxt_msg_last(_NXT_PORT_MSG_CERT_GET), NXT_PORT_MSG_CERT_DELETE = nxt_msg_last(_NXT_PORT_MSG_CERT_DELETE), + NXT_PORT_MSG_SCRIPT_GET = nxt_msg_last(_NXT_PORT_MSG_SCRIPT_GET), + NXT_PORT_MSG_SCRIPT_DELETE = nxt_msg_last(_NXT_PORT_MSG_SCRIPT_DELETE), NXT_PORT_MSG_ACCESS_LOG = nxt_msg_last(_NXT_PORT_MSG_ACCESS_LOG), NXT_PORT_MSG_CHANGE_FILE = nxt_msg_last(_NXT_PORT_MSG_CHANGE_FILE), NXT_PORT_MSG_NEW_PORT = nxt_msg_last(_NXT_PORT_MSG_NEW_PORT), diff --git a/src/nxt_process.c b/src/nxt_process.c index 025efe70..ce2de774 100644 --- a/src/nxt_process.c +++ b/src/nxt_process.c @@ -5,6 +5,8 @@ */ #include <nxt_main.h> + +#include <nxt_application.h> #include <nxt_cgroup.h> #if (NXT_HAVE_LINUX_NS) @@ -651,6 +653,10 @@ nxt_process_setup(nxt_task_t *task, nxt_process_t *process) thread = task->thread; rt = thread->runtime; + if (process->parent_port == rt->port_by_type[NXT_PROCESS_PROTOTYPE]) { + nxt_app_set_logs(); + } + nxt_random_init(&thread->random); rt->type = init->type; @@ -1251,14 +1257,9 @@ nxt_process_close_ports(nxt_task_t *task, nxt_process_t *process) void nxt_process_quit(nxt_task_t *task, nxt_uint_t exit_status) { - nxt_uint_t n; nxt_queue_t *listen; - nxt_runtime_t *rt; nxt_queue_link_t *link, *next; nxt_listen_event_t *lev; - nxt_listen_socket_t *ls; - - rt = task->thread->runtime; nxt_debug(task, "close listen connections"); @@ -1275,21 +1276,5 @@ nxt_process_quit(nxt_task_t *task, nxt_uint_t exit_status) nxt_fd_event_close(task->thread->engine, &lev->socket); } - if (rt->listen_sockets != NULL) { - - ls = rt->listen_sockets->elts; - n = rt->listen_sockets->nelts; - - while (n != 0) { - nxt_socket_close(task, ls->socket); - ls->socket = -1; - - ls++; - n--; - } - - rt->listen_sockets->nelts = 0; - } - nxt_runtime_quit(task, exit_status); } diff --git a/src/nxt_process.h b/src/nxt_process.h index 16d6110c..42fd1bed 100644 --- a/src/nxt_process.h +++ b/src/nxt_process.h @@ -29,6 +29,9 @@ typedef struct { #if (NXT_TLS) nxt_array_t *certs; #endif +#if (NXT_HAVE_NJS) + nxt_array_t *scripts; +#endif } nxt_controller_init_t; diff --git a/src/nxt_router.c b/src/nxt_router.c index 17f6c572..d089cfb8 100644 --- a/src/nxt_router.c +++ b/src/nxt_router.c @@ -11,6 +11,9 @@ #if (NXT_TLS) #include <nxt_cert.h> #endif +#if (NXT_HAVE_NJS) +#include <nxt_script.h> +#endif #include <nxt_http.h> #include <nxt_port_memory_int.h> #include <nxt_unit_request.h> @@ -55,6 +58,17 @@ typedef struct { #endif +#if (NXT_HAVE_NJS) + +typedef struct { + nxt_str_t name; + nxt_router_temp_conf_t *temp_conf; + nxt_queue_link_t link; +} nxt_router_js_module_t; + +#endif + + typedef struct { nxt_str_t *name; nxt_socket_conf_t *socket_conf; @@ -139,6 +153,12 @@ static nxt_int_t nxt_router_conf_tls_insert(nxt_router_temp_conf_t *tmcf, nxt_conf_value_t *value, nxt_socket_conf_t *skcf, nxt_tls_init_t *tls_init, nxt_bool_t last); #endif +#if (NXT_HAVE_NJS) +static void nxt_router_js_module_rpc_handler(nxt_task_t *task, + nxt_port_recv_msg_t *msg, void *data); +static nxt_int_t nxt_router_js_module_insert(nxt_router_temp_conf_t *tmcf, + nxt_conf_value_t *value); +#endif static void nxt_router_app_rpc_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_app_t *app); static void nxt_router_app_prefork_ready(nxt_task_t *task, @@ -1100,6 +1120,10 @@ nxt_router_temp_conf(nxt_task_t *task) nxt_queue_init(&tmcf->tls); #endif +#if (NXT_HAVE_NJS) + nxt_queue_init(&tmcf->js_modules); +#endif + nxt_queue_init(&tmcf->apps); nxt_queue_init(&tmcf->previous); @@ -1154,6 +1178,9 @@ nxt_router_conf_apply(nxt_task_t *task, void *obj, void *data) #if (NXT_TLS) nxt_router_tlssock_t *tls; #endif +#if (NXT_HAVE_NJS) + nxt_router_js_module_t *js_module; +#endif tmcf = obj; @@ -1184,6 +1211,27 @@ nxt_router_conf_apply(nxt_task_t *task, void *obj, void *data) } #endif +#if (NXT_HAVE_NJS) + qlk = nxt_queue_last(&tmcf->js_modules); + + if (qlk != nxt_queue_head(&tmcf->js_modules)) { + nxt_queue_remove(qlk); + + js_module = nxt_queue_link_data(qlk, nxt_router_js_module_t, link); + + nxt_script_store_get(task, &js_module->name, tmcf->mem_pool, + nxt_router_js_module_rpc_handler, js_module); + return; + } +#endif + + rtcf = tmcf->router_conf; + + ret = nxt_tstr_state_done(rtcf->tstr_state, NULL); + if (nxt_slow_path(ret != NXT_OK)) { + goto fail; + } + nxt_queue_each(app, &tmcf->apps, nxt_app_t, link) { if (nxt_router_app_need_start(app)) { @@ -1193,8 +1241,6 @@ nxt_router_conf_apply(nxt_task_t *task, void *obj, void *data) } nxt_queue_loop; - rtcf = tmcf->router_conf; - if (rtcf->access_log != NULL && rtcf->access_log->fd == -1) { nxt_router_access_log_open(task, tmcf); return; @@ -1513,6 +1559,18 @@ static nxt_conf_map_t nxt_router_http_conf[] = { NXT_CONF_MAP_INT8, offsetof(nxt_socket_conf_t, discard_unsafe_fields), }, + + { + nxt_string("log_route"), + NXT_CONF_MAP_INT8, + offsetof(nxt_socket_conf_t, log_route), + }, + + { + nxt_string("server_version"), + NXT_CONF_MAP_INT8, + offsetof(nxt_socket_conf_t, server_version), + }, }; @@ -1558,6 +1616,9 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_tls_init_t *tls_init; nxt_conf_value_t *certificate; #endif +#if (NXT_HAVE_NJS) + nxt_conf_value_t *js_module; +#endif nxt_conf_value_t *root, *conf, *http, *value, *websocket; nxt_conf_value_t *applications, *application; nxt_conf_value_t *listeners, *listener; @@ -1581,6 +1642,9 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, static nxt_str_t conf_timeout_path = nxt_string("/tls/session/timeout"); static nxt_str_t conf_tickets = nxt_string("/tls/session/tickets"); #endif +#if (NXT_HAVE_NJS) + static nxt_str_t js_module_path = nxt_string("/settings/js_module"); +#endif static nxt_str_t static_path = nxt_string("/settings/http/static"); static nxt_str_t websocket_path = nxt_string("/settings/http/websocket"); static nxt_str_t forwarded_path = nxt_string("/forwarded"); @@ -1921,6 +1985,8 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, skcf->proxy_send_timeout = 30 * 1000; skcf->proxy_read_timeout = 30 * 1000; + skcf->server_version = 1; + skcf->websocket_conf.max_frame_size = 1024 * 1024; skcf->websocket_conf.read_timeout = 60 * 1000; skcf->websocket_conf.keepalive_interval = 30 * 1000; @@ -2050,11 +2116,34 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, } } - ret = nxt_tstr_state_done(rtcf->tstr_state, NULL); - if (nxt_slow_path(ret != NXT_OK)) { - goto fail; +#if (NXT_HAVE_NJS) + js_module = nxt_conf_get_path(root, &js_module_path); + + if (js_module != NULL) { + if (nxt_conf_type(js_module) == NXT_CONF_ARRAY) { + n = nxt_conf_array_elements_count(js_module); + + for (i = 0; i < n; i++) { + value = nxt_conf_get_array_element(js_module, i); + + ret = nxt_router_js_module_insert(tmcf, value); + if (nxt_slow_path(ret != NXT_OK)) { + goto fail; + } + } + + } else { + /* NXT_CONF_STRING */ + + ret = nxt_router_js_module_insert(tmcf, js_module); + if (nxt_slow_path(ret != NXT_OK)) { + goto fail; + } + } } +#endif + nxt_queue_add(&deleting_sockets, &router->sockets); nxt_queue_init(&router->sockets); @@ -2106,6 +2195,79 @@ nxt_router_conf_tls_insert(nxt_router_temp_conf_t *tmcf, #endif +#if (NXT_HAVE_NJS) + +static void +nxt_router_js_module_rpc_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg, + void *data) +{ + nxt_int_t ret; + nxt_str_t text; + nxt_router_conf_t *rtcf; + nxt_router_temp_conf_t *tmcf; + nxt_router_js_module_t *js_module; + + nxt_debug(task, "auto module rpc handler"); + + js_module = data; + tmcf = js_module->temp_conf; + + if (msg == NULL || msg->port_msg.type == _NXT_PORT_MSG_RPC_ERROR) { + goto fail; + } + + rtcf = tmcf->router_conf; + + ret = nxt_script_file_read(msg->fd[0], &text); + + nxt_fd_close(msg->fd[0]); + + if (nxt_slow_path(ret == NXT_ERROR)) { + goto fail; + } + + if (text.length > 0) { + ret = nxt_js_add_module(rtcf->tstr_state->jcf, &js_module->name, &text); + + nxt_free(text.start); + + if (nxt_slow_path(ret == NXT_ERROR)) { + goto fail; + } + } + + nxt_work_queue_add(&task->thread->engine->fast_work_queue, + nxt_router_conf_apply, task, tmcf, NULL); + return; + +fail: + + nxt_router_conf_error(task, tmcf); +} + + +static nxt_int_t +nxt_router_js_module_insert(nxt_router_temp_conf_t *tmcf, + nxt_conf_value_t *value) +{ + nxt_router_js_module_t *js_module; + + js_module = nxt_mp_get(tmcf->mem_pool, sizeof(nxt_router_js_module_t)); + if (nxt_slow_path(js_module == NULL)) { + return NXT_ERROR; + } + + js_module->temp_conf = tmcf; + nxt_conf_get_string(value, &js_module->name); + + nxt_queue_insert_tail(&tmcf->js_modules, &js_module->link); + + return NXT_OK; +} + +#endif + + static nxt_int_t nxt_router_conf_process_static(nxt_task_t *task, nxt_router_conf_t *rtcf, nxt_conf_value_t *conf) @@ -3686,6 +3848,13 @@ nxt_router_listen_socket_close(nxt_task_t *task, void *obj, void *data) static void nxt_router_listen_socket_release(nxt_task_t *task, nxt_socket_conf_t *skcf) { +#if (NXT_HAVE_UNIX_DOMAIN) + size_t size; + nxt_buf_t *b; + nxt_port_t *main_port; + nxt_runtime_t *rt; + nxt_sockaddr_t *sa; +#endif nxt_listen_socket_t *ls; nxt_thread_spinlock_t *lock; @@ -3703,10 +3872,38 @@ nxt_router_listen_socket_release(nxt_task_t *task, nxt_socket_conf_t *skcf) nxt_thread_spin_unlock(lock); - if (ls != NULL) { - nxt_socket_close(task, ls->socket); - nxt_free(ls); + if (ls == NULL) { + return; } + + nxt_socket_close(task, ls->socket); + +#if (NXT_HAVE_UNIX_DOMAIN) + sa = ls->sockaddr; + if (sa->u.sockaddr.sa_family != AF_UNIX + || sa->u.sockaddr_un.sun_path[0] == '\0') + { + goto out_free_ls; + } + + size = nxt_sockaddr_size(ls->sockaddr); + + b = nxt_buf_mem_alloc(task->thread->engine->mem_pool, size, 0); + if (b == NULL) { + goto out_free_ls; + } + + b->mem.free = nxt_cpymem(b->mem.free, ls->sockaddr, size); + + rt = task->thread->runtime; + main_port = rt->port_by_type[NXT_PROCESS_MAIN]; + + (void) nxt_port_socket_write(task, main_port, NXT_PORT_MSG_SOCKET_UNLINK, + -1, 0, 0, b); + +out_free_ls: +#endif + nxt_free(ls); } @@ -5206,8 +5403,9 @@ nxt_router_prepare_msg(nxt_task_t *task, nxt_http_request_t *r, req_size = sizeof(nxt_unit_request_t) + r->method->length + 1 + r->version.length + 1 - + r->remote->length + 1 - + r->local->length + 1 + + r->remote->address_length + 1 + + r->local->address_length + 1 + + nxt_sockaddr_port_length(r->local) + 1 + r->server_name.length + 1 + r->target.length + 1 + (r->path->start != r->target.start ? r->path->length + 1 : 0); diff --git a/src/nxt_router.h b/src/nxt_router.h index 11094960..b14f8410 100644 --- a/src/nxt_router.h +++ b/src/nxt_router.h @@ -74,6 +74,10 @@ typedef struct { nxt_queue_t tls; /* of nxt_router_tlssock_t */ #endif +#if (NXT_HAVE_NJS) + nxt_queue_t js_modules; +#endif + nxt_queue_t apps; /* of nxt_app_t */ nxt_queue_t previous; /* of nxt_app_t */ @@ -197,8 +201,12 @@ typedef struct { nxt_str_t body_temp_path; + uint8_t log_route; /* 1 bit */ + uint8_t discard_unsafe_fields; /* 1 bit */ + uint8_t server_version; /* 1 bit */ + nxt_http_forward_t *forwarded; nxt_http_forward_t *client_ip; diff --git a/src/nxt_runtime.c b/src/nxt_runtime.c index c7e4455e..96f801fb 100644 --- a/src/nxt_runtime.c +++ b/src/nxt_runtime.c @@ -563,6 +563,7 @@ nxt_runtime_exit(nxt_task_t *task, void *obj, void *data) #if (NXT_HAVE_UNIX_DOMAIN) { + size_t i; nxt_sockaddr_t *sa; nxt_file_name_t *name; @@ -572,6 +573,22 @@ nxt_runtime_exit(nxt_task_t *task, void *obj, void *data) name = (nxt_file_name_t *) sa->u.sockaddr_un.sun_path; (void) nxt_file_delete(name); } + + for (i = 0; i < rt->listen_sockets->nelts; i++) { + nxt_listen_socket_t *ls; + + ls = (nxt_listen_socket_t *) rt->listen_sockets->elts + i; + sa = ls->sockaddr; + + if (sa->u.sockaddr.sa_family != AF_UNIX + || sa->u.sockaddr_un.sun_path[0] == '\0') + { + continue; + } + + name = (nxt_file_name_t *) sa->u.sockaddr_un.sun_path; + (void) nxt_file_delete(name); + } } #endif } @@ -768,10 +785,10 @@ nxt_runtime_conf_init(nxt_task_t *task, nxt_runtime_t *rt) rt->group = NXT_GROUP; rt->pid = NXT_PID; rt->log = NXT_LOG; - rt->modules = NXT_MODULES; - rt->state = NXT_STATE; + rt->modules = NXT_MODULESDIR; + rt->state = NXT_STATEDIR; rt->control = NXT_CONTROL_SOCK; - rt->tmp = NXT_TMP; + rt->tmp = NXT_TMPDIR; nxt_memzero(&rt->capabilities, sizeof(nxt_capabilities_t)); @@ -889,6 +906,23 @@ nxt_runtime_conf_init(nxt_task_t *task, nxt_runtime_t *rt) "mkdir(%s) failed %E", file_name.start, nxt_errno); } + ret = nxt_file_name_create(rt->mem_pool, &file_name, "%s%sscripts/%Z", + rt->state, slash); + if (nxt_slow_path(ret != NXT_OK)) { + return NXT_ERROR; + } + + ret = mkdir((char *) file_name.start, S_IRWXU); + + if (nxt_fast_path(ret == 0 || nxt_errno == EEXIST)) { + rt->scripts.length = file_name.len; + rt->scripts.start = file_name.start; + + } else { + nxt_alert(task, "Unable to create scripts storage directory: " + "mkdir(%s) failed %E", file_name.start, nxt_errno); + } + control.length = nxt_strlen(rt->control); control.start = (u_char *) rt->control; @@ -927,9 +961,10 @@ nxt_runtime_conf_read_cmd(nxt_task_t *task, nxt_runtime_t *rt) static const char no_pid[] = "option \"--pid\" requires filename\n"; static const char no_log[] = "option \"--log\" requires filename\n"; static const char no_modules[] = - "option \"--modules\" requires directory\n"; - static const char no_state[] = "option \"--state\" requires directory\n"; - static const char no_tmp[] = "option \"--tmp\" requires directory\n"; + "option \"--modulesdir\" requires directory\n"; + static const char no_state[] = + "option \"--statedir\" requires directory\n"; + static const char no_tmp[] = "option \"--tmpdir\" requires directory\n"; static const char help[] = "\n" @@ -948,14 +983,14 @@ nxt_runtime_conf_read_cmd(nxt_task_t *task, nxt_runtime_t *rt) " --log FILE set log filename\n" " default: \"" NXT_LOG "\"\n" "\n" - " --modules DIRECTORY set modules directory name\n" - " default: \"" NXT_MODULES "\"\n" + " --modulesdir DIR set modules directory name\n" + " default: \"" NXT_MODULESDIR "\"\n" "\n" - " --state DIRECTORY set state directory name\n" - " default: \"" NXT_STATE "\"\n" + " --statedir DIR set state directory name\n" + " default: \"" NXT_STATEDIR "\"\n" "\n" - " --tmp DIRECTORY set tmp directory name\n" - " default: \"" NXT_TMP "\"\n" + " --tmpdir DIR set tmp directory name\n" + " default: \"" NXT_TMPDIR "\"\n" "\n" " --user USER set non-privileged processes to run" " as specified user\n" @@ -1038,7 +1073,7 @@ nxt_runtime_conf_read_cmd(nxt_task_t *task, nxt_runtime_t *rt) continue; } - if (nxt_strcmp(p, "--modules") == 0) { + if (nxt_strcmp(p, "--modulesdir") == 0) { if (*argv == NULL) { write(STDERR_FILENO, no_modules, nxt_length(no_modules)); return NXT_ERROR; @@ -1051,7 +1086,7 @@ nxt_runtime_conf_read_cmd(nxt_task_t *task, nxt_runtime_t *rt) continue; } - if (nxt_strcmp(p, "--state") == 0) { + if (nxt_strcmp(p, "--statedir") == 0) { if (*argv == NULL) { write(STDERR_FILENO, no_state, nxt_length(no_state)); return NXT_ERROR; @@ -1064,7 +1099,7 @@ nxt_runtime_conf_read_cmd(nxt_task_t *task, nxt_runtime_t *rt) continue; } - if (nxt_strcmp(p, "--tmp") == 0) { + if (nxt_strcmp(p, "--tmpdir") == 0) { if (*argv == NULL) { write(STDERR_FILENO, no_tmp, nxt_length(no_tmp)); return NXT_ERROR; diff --git a/src/nxt_runtime.h b/src/nxt_runtime.h index 687914f0..66ec0106 100644 --- a/src/nxt_runtime.h +++ b/src/nxt_runtime.h @@ -74,6 +74,7 @@ struct nxt_runtime_s { const char *tmp; nxt_str_t certs; + nxt_str_t scripts; nxt_queue_t engines; /* of nxt_event_engine_t */ diff --git a/src/nxt_script.c b/src/nxt_script.c new file mode 100644 index 00000000..70045a22 --- /dev/null +++ b/src/nxt_script.c @@ -0,0 +1,709 @@ + +/* + * Copyright (C) NGINX, Inc. + * Copyright (C) Zhidao HONG + */ + +#include <nxt_main.h> +#include <nxt_conf.h> +#include <nxt_script.h> +#include <dirent.h> + + +struct nxt_script_s { + nxt_str_t text; +}; + + +typedef struct { + nxt_str_t name; + nxt_conf_value_t *value; + nxt_mp_t *mp; +} nxt_script_info_t; + + +typedef struct { + nxt_str_t name; + nxt_fd_t fd; +} nxt_script_item_t; + + +static nxt_script_t *nxt_script_get(nxt_task_t *task, nxt_str_t *name, + nxt_fd_t fd); +static nxt_conf_value_t *nxt_script_details(nxt_mp_t *mp, nxt_script_t *cert); +static void nxt_script_buf_completion(nxt_task_t *task, void *obj, void *data); + + +static nxt_lvlhsh_t nxt_script_info; + + +static njs_vm_ops_t nxt_js_ops = { + NULL, + NULL, + nxt_js_module_loader, + NULL, +}; + + +nxt_script_t * +nxt_script_new(nxt_task_t *task, nxt_str_t *name, u_char *data, size_t size, + u_char *error) +{ + u_char *start; + njs_vm_t *vm; + njs_str_t mod_name; + njs_mod_t *mod; + njs_vm_opt_t opts; + nxt_script_t *script; + + njs_vm_opt_init(&opts); + + opts.backtrace = 1; + + opts.file.start = (u_char *) "default"; + opts.file.length = 7; + + opts.ops = &nxt_js_ops; + + vm = njs_vm_create(&opts); + if (nxt_slow_path(vm == NULL)) { + return NULL; + } + + mod_name.length = name->length; + mod_name.start = name->start; + + start = data; + + mod = njs_vm_compile_module(vm, &mod_name, &start, start + size); + + if (nxt_slow_path(mod == NULL)) { + (void) nxt_js_error(vm, error); + nxt_alert(task, "JS compile module(%V) failed: %s", name, error); + + goto fail; + } + + script = nxt_zalloc(sizeof(nxt_script_t) + size); + if (nxt_slow_path(script == NULL)) { + goto fail; + } + + script->text.length = size; + script->text.start = (u_char *) script + sizeof(nxt_script_t); + + nxt_memcpy(script->text.start, data, size); + + njs_vm_destroy(vm); + + return script; + +fail: + + njs_vm_destroy(vm); + + return NULL; +} + + +static nxt_script_t * +nxt_script_get(nxt_task_t *task, nxt_str_t *name, nxt_fd_t fd) +{ + nxt_int_t ret; + nxt_str_t text; + nxt_script_t *script; + u_char error[NXT_MAX_ERROR_STR]; + + ret = nxt_script_file_read(fd, &text); + if (nxt_slow_path(ret != NXT_OK)) { + return NULL; + } + + script = nxt_script_new(task, name, text.start, text.length, error); + + nxt_free(text.start); + + return script; +} + + +void +nxt_script_destroy(nxt_script_t *script) +{ + nxt_free(script); +} + + +static nxt_int_t +nxt_script_info_hash_test(nxt_lvlhsh_query_t *lhq, void *data) +{ + nxt_script_info_t *info; + + info = data; + + if (nxt_strcasestr_eq(&lhq->key, &info->name)) { + return NXT_OK; + } + + return NXT_DECLINED; +} + + +static const nxt_lvlhsh_proto_t nxt_script_info_hash_proto + nxt_aligned(64) = +{ + NXT_LVLHSH_DEFAULT, + nxt_script_info_hash_test, + nxt_lvlhsh_alloc, + nxt_lvlhsh_free, +}; + + +void +nxt_script_info_init(nxt_task_t *task, nxt_array_t *scripts) +{ + uint32_t i; + nxt_script_t *script; + nxt_script_item_t *item; + + item = scripts->elts; + + for (i = 0; i < scripts->nelts; i++) { + script = nxt_script_get(task, &item->name, item->fd); + + if (nxt_slow_path(script == NULL)) { + continue; + } + + (void) nxt_script_info_save(&item->name, script); + + nxt_script_destroy(script); + + item++; + } +} + + +nxt_int_t +nxt_script_info_save(nxt_str_t *name, nxt_script_t *script) +{ + nxt_mp_t *mp; + nxt_int_t ret; + nxt_conf_value_t *value; + nxt_script_info_t *info; + nxt_lvlhsh_query_t lhq; + + mp = nxt_mp_create(1024, 128, 256, 32); + if (nxt_slow_path(mp == NULL)) { + return NXT_ERROR; + } + + info = nxt_mp_get(mp, sizeof(nxt_script_info_t)); + if (nxt_slow_path(info == NULL)) { + goto fail; + } + + name = nxt_str_dup(mp, &info->name, name); + if (nxt_slow_path(name == NULL)) { + goto fail; + } + + value = nxt_script_details(mp, script); + if (nxt_slow_path(value == NULL)) { + goto fail; + } + + info->mp = mp; + info->value = value; + + lhq.key_hash = nxt_djb_hash(name->start, name->length); + lhq.replace = 1; + lhq.key = *name; + lhq.value = info; + lhq.proto = &nxt_script_info_hash_proto; + + ret = nxt_lvlhsh_insert(&nxt_script_info, &lhq); + if (nxt_slow_path(ret != NXT_OK)) { + goto fail; + } + + if (lhq.value != info) { + info = lhq.value; + nxt_mp_destroy(info->mp); + } + + return NXT_OK; + +fail: + + nxt_mp_destroy(mp); + return NXT_ERROR; +} + + +nxt_conf_value_t * +nxt_script_info_get(nxt_str_t *name) +{ + nxt_int_t ret; + nxt_script_info_t *info; + nxt_lvlhsh_query_t lhq; + + lhq.key_hash = nxt_djb_hash(name->start, name->length); + lhq.key = *name; + lhq.proto = &nxt_script_info_hash_proto; + + ret = nxt_lvlhsh_find(&nxt_script_info, &lhq); + if (ret != NXT_OK) { + return NULL; + } + + info = lhq.value; + + return info->value; +} + + +nxt_conf_value_t * +nxt_script_info_get_all(nxt_mp_t *mp) +{ + uint32_t i; + nxt_conf_value_t *all; + nxt_script_info_t *info; + nxt_lvlhsh_each_t lhe; + + nxt_lvlhsh_each_init(&lhe, &nxt_script_info_hash_proto); + + for (i = 0; /* void */; i++) { + info = nxt_lvlhsh_each(&nxt_script_info, &lhe); + + if (info == NULL) { + break; + } + } + + all = nxt_conf_create_object(mp, i); + if (nxt_slow_path(all == NULL)) { + return NULL; + } + + nxt_lvlhsh_each_init(&lhe, &nxt_script_info_hash_proto); + + for (i = 0; /* void */; i++) { + info = nxt_lvlhsh_each(&nxt_script_info, &lhe); + + if (info == NULL) { + break; + } + + nxt_conf_set_member(all, &info->name, info->value, i); + } + + return all; +} + + +static nxt_conf_value_t * +nxt_script_details(nxt_mp_t *mp, nxt_script_t *script) +{ + nxt_conf_value_t *value; + + value = nxt_conf_create_object(mp, 0); + if (nxt_slow_path(value == NULL)) { + return NULL; + } + + nxt_conf_set_string_dup(value, mp, &script->text); + + return value; +} + + +nxt_int_t +nxt_script_info_delete(nxt_str_t *name) +{ + nxt_int_t ret; + nxt_script_info_t *info; + nxt_lvlhsh_query_t lhq; + + lhq.key_hash = nxt_djb_hash(name->start, name->length); + lhq.key = *name; + lhq.proto = &nxt_script_info_hash_proto; + + ret = nxt_lvlhsh_delete(&nxt_script_info, &lhq); + + if (ret == NXT_OK) { + info = lhq.value; + nxt_mp_destroy(info->mp); + } + + return ret; +} + + +nxt_array_t * +nxt_script_store_load(nxt_task_t *task, nxt_mp_t *mp) +{ + DIR *dir; + size_t size, alloc; + u_char *buf, *p; + nxt_str_t name; + nxt_int_t ret; + nxt_file_t file; + nxt_array_t *scripts; + nxt_runtime_t *rt; + struct dirent *de; + nxt_script_item_t *item; + + rt = task->thread->runtime; + + if (nxt_slow_path(rt->scripts.start == NULL)) { + nxt_alert(task, "no scripts storage directory"); + return NULL; + } + + scripts = nxt_array_create(mp, 16, sizeof(nxt_script_item_t)); + if (nxt_slow_path(scripts == NULL)) { + return NULL; + } + + buf = NULL; + alloc = 0; + + dir = opendir((char *) rt->scripts.start); + if (nxt_slow_path(dir == NULL)) { + nxt_alert(task, "opendir(\"%s\") failed %E", + rt->scripts.start, nxt_errno); + goto fail; + } + + for ( ;; ) { + de = readdir(dir); + if (de == NULL) { + break; + } + + nxt_debug(task, "readdir(\"%s\"): \"%s\"", + rt->scripts.start, de->d_name); + + name.length = nxt_strlen(de->d_name); + name.start = (u_char *) de->d_name; + + if (nxt_str_eq(&name, ".", 1) || nxt_str_eq(&name, "..", 2)) { + continue; + } + + item = nxt_array_add(scripts); + if (nxt_slow_path(item == NULL)) { + goto fail; + } + + item->fd = -1; + + size = rt->scripts.length + name.length + 1; + + if (size > alloc) { + size += 32; + + p = nxt_realloc(buf, size); + if (p == NULL) { + goto fail; + } + + alloc = size; + buf = p; + } + + p = nxt_cpymem(buf, rt->scripts.start, rt->scripts.length); + p = nxt_cpymem(p, name.start, name.length + 1); + + nxt_memzero(&file, sizeof(nxt_file_t)); + + file.name = buf; + + ret = nxt_file_open(task, &file, NXT_FILE_RDONLY, NXT_FILE_OPEN, + NXT_FILE_OWNER_ACCESS); + + if (nxt_slow_path(ret != NXT_OK)) { + nxt_array_remove_last(scripts); + continue; + } + + item->fd = file.fd; + + if (nxt_slow_path(nxt_str_dup(mp, &item->name, &name) == NULL)) { + goto fail; + } + } + + if (buf != NULL) { + nxt_free(buf); + } + + (void) closedir(dir); + + return scripts; + +fail: + + if (buf != NULL) { + nxt_free(buf); + } + + if (dir != NULL) { + (void) closedir(dir); + } + + nxt_script_store_release(scripts); + + return NULL; +} + + +void +nxt_script_store_release(nxt_array_t *scripts) +{ + uint32_t i; + nxt_script_item_t *item; + + item = scripts->elts; + + for (i = 0; i < scripts->nelts; i++) { + nxt_fd_close(item[i].fd); + } + + nxt_array_destroy(scripts); +} + + +void +nxt_script_store_get(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp, + nxt_port_rpc_handler_t handler, void *ctx) +{ + uint32_t stream; + nxt_int_t ret; + nxt_buf_t *b; + nxt_port_t *main_port, *recv_port; + nxt_runtime_t *rt; + + b = nxt_buf_mem_alloc(mp, name->length + 1, 0); + if (nxt_slow_path(b == NULL)) { + goto fail; + } + + nxt_mp_retain(mp); + b->completion_handler = nxt_script_buf_completion; + + nxt_buf_cpystr(b, name); + *b->mem.free++ = '\0'; + + rt = task->thread->runtime; + main_port = rt->port_by_type[NXT_PROCESS_MAIN]; + recv_port = rt->port_by_type[rt->type]; + + stream = nxt_port_rpc_register_handler(task, recv_port, handler, handler, + -1, ctx); + if (nxt_slow_path(stream == 0)) { + goto fail; + } + + ret = nxt_port_socket_write(task, main_port, NXT_PORT_MSG_SCRIPT_GET, -1, + stream, recv_port->id, b); + + if (nxt_slow_path(ret != NXT_OK)) { + nxt_port_rpc_cancel(task, recv_port, stream); + goto fail; + } + + return; + +fail: + + handler(task, NULL, ctx); +} + + +static void +nxt_script_buf_completion(nxt_task_t *task, void *obj, void *data) +{ + nxt_mp_t *mp; + nxt_buf_t *b; + + b = obj; + mp = b->data; + nxt_assert(b->next == NULL); + + nxt_mp_free(mp, b); + nxt_mp_release(mp); +} + + +void +nxt_script_store_get_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg) +{ + u_char *p; + nxt_int_t ret; + nxt_str_t name; + nxt_file_t file; + nxt_port_t *port; + nxt_runtime_t *rt; + nxt_port_msg_type_t type; + + port = nxt_runtime_port_find(task->thread->runtime, msg->port_msg.pid, + msg->port_msg.reply_port); + + if (nxt_slow_path(port == NULL)) { + nxt_alert(task, "process port not found (pid %PI, reply_port %d)", + msg->port_msg.pid, msg->port_msg.reply_port); + return; + } + + if (nxt_slow_path(port->type != NXT_PROCESS_CONTROLLER + && port->type != NXT_PROCESS_ROUTER)) + { + nxt_alert(task, "process %PI cannot store scripts", + msg->port_msg.pid); + return; + } + + nxt_memzero(&file, sizeof(nxt_file_t)); + + file.fd = -1; + type = NXT_PORT_MSG_RPC_ERROR; + + rt = task->thread->runtime; + + if (nxt_slow_path(rt->certs.start == NULL)) { + nxt_alert(task, "no scripts storage directory"); + goto error; + } + + name.start = msg->buf->mem.pos; + name.length = nxt_strlen(name.start); + + file.name = nxt_malloc(rt->scripts.length + name.length + 1); + if (nxt_slow_path(file.name == NULL)) { + goto error; + } + + p = nxt_cpymem(file.name, rt->scripts.start, rt->scripts.length); + p = nxt_cpymem(p, name.start, name.length + 1); + + ret = nxt_file_open(task, &file, NXT_FILE_RDWR, NXT_FILE_CREATE_OR_OPEN, + NXT_FILE_OWNER_ACCESS); + + nxt_free(file.name); + + if (nxt_fast_path(ret == NXT_OK)) { + type = NXT_PORT_MSG_RPC_READY_LAST | NXT_PORT_MSG_CLOSE_FD; + } + +error: + + (void) nxt_port_socket_write(task, port, type, file.fd, + msg->port_msg.stream, 0, NULL); +} + + +void +nxt_script_store_delete(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp) +{ + nxt_buf_t *b; + nxt_port_t *main_port; + nxt_runtime_t *rt; + + b = nxt_buf_mem_alloc(mp, name->length + 1, 0); + + if (nxt_fast_path(b != NULL)) { + nxt_buf_cpystr(b, name); + *b->mem.free++ = '\0'; + + rt = task->thread->runtime; + main_port = rt->port_by_type[NXT_PROCESS_MAIN]; + + (void) nxt_port_socket_write(task, main_port, + NXT_PORT_MSG_SCRIPT_DELETE, -1, 0, 0, b); + } +} + + +void +nxt_script_store_delete_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg) +{ + u_char *p; + nxt_str_t name; + nxt_port_t *ctl_port; + nxt_runtime_t *rt; + nxt_file_name_t *path; + + rt = task->thread->runtime; + ctl_port = rt->port_by_type[NXT_PROCESS_CONTROLLER]; + + if (nxt_slow_path(ctl_port == NULL)) { + nxt_alert(task, "controller port not found"); + return; + } + + if (nxt_slow_path(nxt_recv_msg_cmsg_pid(msg) != ctl_port->pid)) { + nxt_alert(task, "process %PI cannot delete scripts", + nxt_recv_msg_cmsg_pid(msg)); + return; + } + + if (nxt_slow_path(rt->scripts.start == NULL)) { + nxt_alert(task, "no scripts storage directory"); + return; + } + + name.start = msg->buf->mem.pos; + name.length = nxt_strlen(name.start); + + path = nxt_malloc(rt->scripts.length + name.length + 1); + + if (nxt_fast_path(path != NULL)) { + p = nxt_cpymem(path, rt->scripts.start, rt->scripts.length); + p = nxt_cpymem(p, name.start, name.length + 1); + + (void) nxt_file_delete(path); + + nxt_free(path); + } +} + + +nxt_int_t +nxt_script_file_read(nxt_fd_t fd, nxt_str_t *str) +{ + ssize_t n; + nxt_int_t ret; + nxt_file_t file; + nxt_file_info_t fi; + + nxt_memzero(&file, sizeof(nxt_file_t)); + + file.fd = fd; + + ret = nxt_file_info(&file, &fi); + if (nxt_slow_path(ret != NXT_OK)) { + return NXT_ERROR; + } + + if (nxt_slow_path(!nxt_is_file(&fi))) { + nxt_str_null(str); + return NXT_DECLINED; + } + + str->length = nxt_file_size(&fi); + str->start = nxt_malloc(str->length); + if (nxt_slow_path(str->start == NULL)) { + return NXT_ERROR; + } + + n = nxt_file_read(&file, str->start, str->length, 0); + + if (nxt_slow_path(n != (ssize_t) str->length)) { + nxt_free(str->start); + return NXT_ERROR; + } + + return NXT_OK; +} diff --git a/src/nxt_script.h b/src/nxt_script.h new file mode 100644 index 00000000..ffefc108 --- /dev/null +++ b/src/nxt_script.h @@ -0,0 +1,37 @@ + +/* + * Copyright (C) NGINX, Inc. + * Copyright (C) Zhidao HONG + */ + +#ifndef _NXT_SCRIPT_INCLUDED_ +#define _NXT_SCRIPT_INCLUDED_ + + +typedef struct nxt_script_s nxt_script_t; + +nxt_script_t *nxt_script_new(nxt_task_t *task, nxt_str_t *name, u_char *data, + size_t size, u_char *error); +void nxt_script_destroy(nxt_script_t *script); + +void nxt_script_info_init(nxt_task_t *task, nxt_array_t *scripts); +nxt_int_t nxt_script_info_save(nxt_str_t *name, nxt_script_t *script); +nxt_conf_value_t *nxt_script_info_get(nxt_str_t *name); +nxt_conf_value_t *nxt_script_info_get_all(nxt_mp_t *mp); +nxt_int_t nxt_script_info_delete(nxt_str_t *name); + +nxt_array_t *nxt_script_store_load(nxt_task_t *task, nxt_mp_t *mem_pool); +void nxt_script_store_release(nxt_array_t *scripts); + +void nxt_script_store_get(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp, + nxt_port_rpc_handler_t handler, void *ctx); +void nxt_script_store_delete(nxt_task_t *task, nxt_str_t *name, nxt_mp_t *mp); + +void nxt_script_store_get_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg); +void nxt_script_store_delete_handler(nxt_task_t *task, + nxt_port_recv_msg_t *msg); + +nxt_int_t nxt_script_file_read(nxt_fd_t fd, nxt_str_t *str); + + +#endif /* _NXT_SCRIPT_INCLUDED_ */ diff --git a/src/nxt_stream_module.c b/src/nxt_stream_module.c deleted file mode 100644 index 25aaec57..00000000 --- a/src/nxt_stream_module.c +++ /dev/null @@ -1,131 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> -#include <nxt_runtime.h> - - -static void nxt_stream_connection_peer(nxt_task_t *task, - nxt_upstream_peer_t *up); -static void nxt_stream_connection_close(nxt_task_t *task, void *obj, - void *data); - - -void -nxt_stream_connection_init(nxt_task_t *task, void *obj, void *data) -{ - nxt_conn_t *c; - nxt_runtime_t *rt; - nxt_upstream_peer_t *up; - - c = obj; - - nxt_debug(task, "stream connection init"); - - up = nxt_mp_zget(c->mem_pool, sizeof(nxt_upstream_peer_t)); - if (nxt_slow_path(up == NULL)) { - goto fail; - } - - up->data = c; - - rt = task->thread->runtime; - - if (rt->upstream.length != 0) { - up->addr = rt->upstream; - - } else { - nxt_str_set(&up->addr, "127.0.0.1:8080"); - } - - up->ready_handler = nxt_stream_connection_peer; - up->mem_pool = c->mem_pool; - - nxt_upstream_round_robin_peer(task, up); - return; - -fail: - - /* TODO: close connection */ - return; -} - - -static void -nxt_stream_connection_peer(nxt_task_t *task, nxt_upstream_peer_t *up) -{ - nxt_conn_t *c; - nxt_conn_proxy_t *p; - - c = up->data; - - up->sockaddr->type = SOCK_STREAM; - - nxt_log_debug(c->socket.log, "stream connection peer %*s", - (size_t) up->sockaddr->length, - nxt_sockaddr_start(up->sockaddr)); - - p = nxt_conn_proxy_create(c); - if (nxt_slow_path(p == NULL)) { - goto fail; - } - - p->client->socket.data = p; - p->peer->socket.data = p; - - p->client_buffer_size = 1024; - p->peer_buffer_size = 4096; - //p->client_wait_timeout = 9000; - p->connect_timeout = 7000; - p->reconnect_timeout = 500; - //p->peer_wait_timeout = 5000; - p->client_write_timeout = 3000; - p->peer_write_timeout = 3000; - p->completion_handler = nxt_stream_connection_close; - //p->retries = 10; - p->peer->remote = up->sockaddr; - - if (0) { - nxt_event_engine_t *engine; - nxt_event_write_rate_t *rate; - - rate = nxt_mp_get(c->mem_pool, sizeof(nxt_event_write_rate_t)); - - if (nxt_slow_path(rate == NULL)) { - goto fail; - } - - c->rate = rate; - - rate->limit = 1024; - rate->limit_after = 0; - rate->average = rate->limit; - - engine = nxt_thread_event_engine(); - rate->last = engine->timers.now; - } - - nxt_conn_proxy(task, p); - return; - -fail: - - /* TODO: close connection */ - return; -} - - -static void -nxt_stream_connection_close(nxt_task_t *task, void *obj, void *data) -{ - nxt_event_conn_proxy_t *p; - - p = obj; - - nxt_log_debug(p->client->socket.log, "stream connection close"); - - nxt_mp_destroy(p->client->mem_pool); -} diff --git a/src/nxt_stream_source.c b/src/nxt_stream_source.c deleted file mode 100644 index 66ec1640..00000000 --- a/src/nxt_stream_source.c +++ /dev/null @@ -1,480 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -static void nxt_stream_source_connected(nxt_task_t *task, void *obj, - void *data); -static void nxt_stream_source_write_ready(nxt_task_t *task, void *obj, - void *data); -static void nxt_stream_source_read_ready(nxt_task_t *task, void *obj, - void *data); -static nxt_buf_t *nxt_stream_source_process_buffers(nxt_stream_source_t *stream, - nxt_event_conn_t *c); -static void nxt_stream_source_buf_completion(nxt_task_t *task, void *obj, - void *data); -static void nxt_stream_source_read_done(nxt_task_t *task, void *obj, - void *data); -static void nxt_stream_source_refused(nxt_task_t *task, void *obj, void *data); -static void nxt_stream_source_closed(nxt_task_t *task, void *obj, void *data); -static void nxt_stream_source_error(nxt_task_t *task, void *obj, void *data); -static void nxt_stream_source_close(nxt_task_t *task, - nxt_stream_source_t *stream); - - -static const nxt_event_conn_state_t nxt_stream_source_connect_state; -static const nxt_event_conn_state_t nxt_stream_source_request_write_state; -static const nxt_event_conn_state_t nxt_stream_source_response_ready_state; -static const nxt_event_conn_state_t nxt_stream_source_response_read_state; - - -void -nxt_stream_source_connect(nxt_task_t *task, nxt_stream_source_t *stream) -{ - nxt_thread_t *thr; - nxt_event_conn_t *c; - nxt_upstream_source_t *us; - - thr = nxt_thread(); - - us = stream->upstream; - - if (nxt_slow_path(!nxt_buf_pool_obtainable(&us->buffers))) { - nxt_log(task, NXT_LOG_ERR, - "%d buffers %uDK each are not enough to read upstream response", - us->buffers.max, us->buffers.size / 1024); - goto fail; - } - - c = nxt_event_conn_create(us->buffers.mem_pool, thr->log); - if (nxt_slow_path(c == NULL)) { - goto fail; - } - - stream->conn = c; - c->socket.data = stream; - - nxt_conn_work_queue_set(c, us->work_queue); - - c->remote = us->peer->sockaddr; - c->write_state = &nxt_stream_source_connect_state; - - nxt_event_conn_connect(task, c); - return; - -fail: - - stream->error_handler(task, stream); -} - - -static const nxt_event_conn_state_t nxt_stream_source_connect_state - nxt_aligned(64) = -{ - NXT_EVENT_NO_BUF_PROCESS, - NXT_EVENT_TIMER_AUTORESET, - - nxt_stream_source_connected, - nxt_stream_source_refused, - nxt_stream_source_error, - - NULL, /* timeout */ - NULL, /* timeout value */ - 0, /* connect_timeout */ -}; - - -static void -nxt_stream_source_connected(nxt_task_t *task, void *obj, void *data) -{ - nxt_event_conn_t *c; - nxt_stream_source_t *stream; - - c = obj; - stream = data; - - nxt_debug(task, "stream source connected fd:%d", c->socket.fd); - - c->read_state = &nxt_stream_source_response_ready_state; - c->write = stream->out; - c->write_state = &nxt_stream_source_request_write_state; - - if (task->thread->engine->batch != 0) { - nxt_event_conn_write(task, c); - - } else { - stream->read_queued = 1; - nxt_thread_work_queue_add(task->thread, - &task->thread->engine->read_work_queue, - c->io->read, task, c, stream); - - c->io->write(task, c, stream); - } -} - - -static const nxt_event_conn_state_t nxt_stream_source_request_write_state - nxt_aligned(64) = -{ - NXT_EVENT_NO_BUF_PROCESS, - NXT_EVENT_TIMER_AUTORESET, - - nxt_stream_source_write_ready, - NULL, - nxt_stream_source_error, - - NULL, /* timeout */ - NULL, /* timeout value */ - 0, /* connect_timeout */ -}; - - -static const nxt_event_conn_state_t nxt_stream_source_response_ready_state - nxt_aligned(64) = -{ - NXT_EVENT_NO_BUF_PROCESS, - NXT_EVENT_TIMER_AUTORESET, - - nxt_stream_source_read_ready, - nxt_stream_source_closed, - nxt_stream_source_error, - - NULL, /* timeout */ - NULL, /* timeout value */ - 0, /* connect_timeout */ -}; - - -static void -nxt_stream_source_write_ready(nxt_task_t *task, void *obj, void *data) -{ - nxt_event_conn_t *c; - - c = obj; - - nxt_debug(task, "stream source write ready fd:%d", c->socket.fd); - - nxt_conn_read(task, c); -} - - -static void -nxt_stream_source_read_ready(nxt_task_t *task, void *obj, void *data) -{ - nxt_int_t ret; - nxt_buf_t *b; - nxt_buf_pool_t *buffers; - nxt_event_conn_t *c; - nxt_stream_source_t *stream; - - c = obj; - stream = data; - stream->read_queued = 0; - - nxt_debug(task, "stream source read ready fd:%d", c->socket.fd); - - if (c->read == NULL) { - - buffers = &stream->upstream->buffers; - - ret = nxt_buf_pool_mem_alloc(buffers, 0); - - if (nxt_slow_path(ret != NXT_OK)) { - - if (nxt_slow_path(ret == NXT_ERROR)) { - goto fail; - } - - /* ret == NXT_AGAIN */ - - nxt_debug(task, "stream source flush"); - - b = nxt_buf_sync_alloc(buffers->mem_pool, NXT_BUF_SYNC_NOBUF); - - if (nxt_slow_path(b == NULL)) { - goto fail; - } - - nxt_event_fd_block_read(task->thread->engine, &c->socket); - - nxt_source_filter(task->thread, c->write_work_queue, task, - stream->next, b); - return; - } - - c->read = buffers->current; - buffers->current = NULL; - } - - c->read_state = &nxt_stream_source_response_read_state; - - nxt_conn_read(task, c); - return; - -fail: - - nxt_stream_source_close(task, stream); -} - - -static const nxt_event_conn_state_t nxt_stream_source_response_read_state - nxt_aligned(64) = -{ - NXT_EVENT_NO_BUF_PROCESS, - NXT_EVENT_TIMER_AUTORESET, - - nxt_stream_source_read_done, - nxt_stream_source_closed, - nxt_stream_source_error, - - NULL, /* timeout */ - NULL, /* timeout value */ - 0, /* connect_timeout */ -}; - - -static void -nxt_stream_source_read_done(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b; - nxt_bool_t batch; - nxt_event_conn_t *c; - nxt_stream_source_t *stream; - - c = obj; - stream = data; - - nxt_debug(task, "stream source read done fd:%d", c->socket.fd); - - if (c->read != NULL) { - b = nxt_stream_source_process_buffers(stream, c); - - if (nxt_slow_path(b == NULL)) { - nxt_stream_source_close(task, stream); - return; - } - - batch = (task->thread->engine->batch != 0); - - if (batch) { - nxt_thread_work_queue_add(task->thread, - stream->upstream->work_queue, - nxt_source_filter_handler, - task, stream->next, b); - } - - if (!stream->read_queued) { - stream->read_queued = 1; - nxt_thread_work_queue_add(task->thread, - stream->upstream->work_queue, - nxt_stream_source_read_ready, - task, c, stream); - } - - if (!batch) { - stream->next->filter(task, stream->next->context, b); - } - } -} - - -static nxt_buf_t * -nxt_stream_source_process_buffers(nxt_stream_source_t *stream, - nxt_event_conn_t *c) -{ - size_t size, nbytes; - nxt_buf_t *b, *in, *head, **prev; - - nbytes = c->nbytes; - prev = &head; - - do { - b = nxt_buf_mem_alloc(stream->upstream->buffers.mem_pool, 0, 0); - - if (nxt_slow_path(b == NULL)) { - return NULL; - } - - *prev = b; - - b->data = stream; - b->completion_handler = nxt_stream_source_buf_completion; - - in = c->read; - in->retain++; - b->parent = in; - - b->mem.pos = in->mem.free; - b->mem.start = in->mem.free; - - size = nxt_buf_mem_free_size(&in->mem); - - if (nbytes < size) { - in->mem.free += nbytes; - - b->mem.free = in->mem.free; - b->mem.end = in->mem.free; - - break; - } - - in->mem.free = in->mem.end; - - b->mem.free = in->mem.free; - b->mem.end = in->mem.free; - nbytes -= size; - - prev = &b->next; - c->read = in->next; - in->next = NULL; - - } while (c->read != NULL); - - return head; -} - - -static void -nxt_stream_source_buf_completion(nxt_task_t *task, void *obj, void *data) -{ - size_t size; - nxt_buf_t *b, *parent; - nxt_stream_source_t *stream; - - b = obj; - parent = data; - -#if 0 - nxt_debug(thr->log, - "stream source buf completion: %p parent:%p retain:%uD", - b, parent, parent->retain); -#endif - - stream = b->data; - - /* A parent is a buffer where stream reads data. */ - - parent->mem.pos = b->mem.pos; - parent->retain--; - - if (parent->retain == 0 && !stream->conn->socket.closed) { - size = nxt_buf_mem_size(&parent->mem); - - parent->mem.pos = parent->mem.start; - parent->mem.free = parent->mem.start; - - /* - * A buffer's original size can be changed by filters - * so reuse the buffer only if it is still large enough. - */ - if (size >= 256 || size >= stream->upstream->buffers.size) { - - if (stream->conn->read != parent) { - nxt_buf_chain_add(&stream->conn->read, parent); - } - - if (!stream->read_queued) { - stream->read_queued = 1; - nxt_thread_work_queue_add(task->thread, - stream->upstream->work_queue, - nxt_stream_source_read_ready, - task, stream->conn, - stream->conn->socket.data); - } - } - } - - nxt_buf_free(stream->upstream->buffers.mem_pool, b); -} - - -static void -nxt_stream_source_refused(nxt_task_t *task, void *obj, void *data) -{ - nxt_stream_source_t *stream; - - stream = data; - -#if (NXT_DEBUG) - { - nxt_event_conn_t *c; - - c = obj; - - nxt_debug(task, "stream source refused fd:%d", c->socket.fd); - } -#endif - - nxt_stream_source_close(task, stream); -} - - -static void -nxt_stream_source_closed(nxt_task_t *task, void *obj, void *data) -{ - nxt_buf_t *b; - nxt_event_conn_t *c; - nxt_stream_source_t *stream; - - c = obj; - stream = data; - - nxt_debug(task, "stream source closed fd:%d", c->socket.fd); - - nxt_conn_close(task, c); - - b = nxt_buf_sync_alloc(stream->upstream->buffers.mem_pool, - NXT_BUF_SYNC_LAST); - - if (nxt_slow_path(b == NULL)) { - stream->error_handler(task, stream); - return; - } - - nxt_source_filter(task->thread, c->write_work_queue, task, stream->next, b); -} - - -static void -nxt_stream_source_error(nxt_task_t *task, void *obj, void *data) -{ - nxt_stream_source_t *stream; - - stream = data; - -#if (NXT_DEBUG) - { - nxt_event_fd_t *ev; - - ev = obj; - - nxt_debug(task, "stream source error fd:%d", ev->fd); - } -#endif - - nxt_stream_source_close(task, stream); -} - - -static void -nxt_stream_source_close(nxt_task_t *task, nxt_stream_source_t *stream) -{ - nxt_conn_close(task, stream->conn); - - stream->error_handler(task, stream); -} - - -void -nxt_source_filter_handler(nxt_task_t *task, void *obj, void *data) -{ - nxt_source_hook_t *next; - - next = obj; - - next->filter(task, next->context, data); -} diff --git a/src/nxt_stream_source.h b/src/nxt_stream_source.h deleted file mode 100644 index 2d57073f..00000000 --- a/src/nxt_stream_source.h +++ /dev/null @@ -1,32 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_STREAM_SOURCE_H_INCLUDED_ -#define _NXT_STREAM_SOURCE_H_INCLUDED_ - - -typedef struct nxt_stream_source_s nxt_stream_source_t; - -typedef void (*nxt_stream_source_handler_t)(nxt_task_t *task, - nxt_stream_source_t *s); - -struct nxt_stream_source_s { - nxt_conn_t *conn; - nxt_source_hook_t *next; - nxt_upstream_source_t *upstream; - - nxt_buf_t *out; - - uint32_t read_queued; /* 1 bit */ - - nxt_stream_source_handler_t error_handler; -}; - - -void nxt_stream_source_connect(nxt_task_t *task, nxt_stream_source_t *stream); - - -#endif /* _NXT_STREAM_SOURCE_H_INCLUDED_ */ diff --git a/src/nxt_tstr.c b/src/nxt_tstr.c index fda585b8..516415d9 100644 --- a/src/nxt_tstr.c +++ b/src/nxt_tstr.c @@ -70,7 +70,7 @@ nxt_tstr_state_new(nxt_mp_t *mp, nxt_bool_t test) } #if (NXT_HAVE_NJS) - state->jcf = nxt_js_conf_new(mp); + state->jcf = nxt_js_conf_new(mp, test); if (nxt_slow_path(state->jcf == NULL)) { return NULL; } @@ -273,7 +273,8 @@ nxt_tstr_query(nxt_task_t *task, nxt_tstr_query_t *query, nxt_tstr_t *tstr, } else { #if (NXT_HAVE_NJS) - ret = nxt_js_call(task, &query->cache->js, tstr->u.js, val, query->ctx); + ret = nxt_js_call(task, query->state->jcf, &query->cache->js, + tstr->u.js, val, query->ctx); if (nxt_slow_path(ret != NXT_OK)) { query->failed = 1; @@ -296,6 +297,13 @@ nxt_tstr_query(nxt_task_t *task, nxt_tstr_query_t *query, nxt_tstr_t *tstr, } +nxt_bool_t +nxt_tstr_query_failed(nxt_tstr_query_t *query) +{ + return query->failed; +} + + void nxt_tstr_query_resolve(nxt_task_t *task, nxt_tstr_query_t *query, void *data, nxt_work_handler_t ready, nxt_work_handler_t error) diff --git a/src/nxt_tstr.h b/src/nxt_tstr.h index ce8e6f3a..afa7f56d 100644 --- a/src/nxt_tstr.h +++ b/src/nxt_tstr.h @@ -52,6 +52,7 @@ nxt_int_t nxt_tstr_query_init(nxt_tstr_query_t **query_p, nxt_mp_t *mp); void nxt_tstr_query(nxt_task_t *task, nxt_tstr_query_t *query, nxt_tstr_t *tstr, nxt_str_t *val); +nxt_bool_t nxt_tstr_query_failed(nxt_tstr_query_t *query); void nxt_tstr_query_resolve(nxt_task_t *task, nxt_tstr_query_t *query, void *data, nxt_work_handler_t ready, nxt_work_handler_t error); void nxt_tstr_query_handle(nxt_task_t *task, nxt_tstr_query_t *query, diff --git a/src/nxt_upstream_source.c b/src/nxt_upstream_source.c deleted file mode 100644 index ee3fc21e..00000000 --- a/src/nxt_upstream_source.c +++ /dev/null @@ -1,71 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#include <nxt_main.h> - - -static nxt_int_t nxt_upstream_header_hash_test(nxt_lvlhsh_query_t *lhq, - void *data); - - -const nxt_lvlhsh_proto_t nxt_upstream_header_hash_proto nxt_aligned(64) = { - NXT_LVLHSH_DEFAULT, - 0, - nxt_upstream_header_hash_test, - nxt_mem_lvlhsh_alloc, - nxt_mem_lvlhsh_free, -}; - - -nxt_int_t -nxt_upstream_header_hash_add(nxt_mp_t *mp, nxt_lvlhsh_t *lh, - const nxt_upstream_name_value_t *unv, nxt_uint_t n) -{ - nxt_lvlhsh_query_t lhq; - - while (n != 0) { - lhq.key_hash = nxt_djb_hash(unv->name, unv->len); - lhq.replace = 1; - lhq.key.len = unv->len; - lhq.key.data = (u_char *) unv->name; - lhq.value = (void *) unv; - lhq.proto = &nxt_upstream_header_hash_proto; - lhq.pool = mp; - - if (nxt_lvlhsh_insert(lh, &lhq) != NXT_OK) { - return NXT_ERROR; - } - - unv++; - n--; - } - - return NXT_OK; -} - - -static nxt_int_t -nxt_upstream_header_hash_test(nxt_lvlhsh_query_t *lhq, void *data) -{ - nxt_upstream_name_value_t *unv; - - unv = data; - - if (lhq->key.len == unv->len - && nxt_memcasecmp(lhq->key.data, unv->name, unv->len) == 0) - { - return NXT_OK; - } - - return NXT_DECLINED; -} - - -nxt_int_t -nxt_upstream_name_value_ignore(nxt_upstream_source_t *us, nxt_name_value_t *nv) -{ - return NXT_OK; -} diff --git a/src/nxt_upstream_source.h b/src/nxt_upstream_source.h deleted file mode 100644 index 143b8d0c..00000000 --- a/src/nxt_upstream_source.h +++ /dev/null @@ -1,83 +0,0 @@ - -/* - * Copyright (C) Igor Sysoev - * Copyright (C) NGINX, Inc. - */ - -#ifndef _NXT_UPSTREAM_SOURCE_H_INCLUDED_ -#define _NXT_UPSTREAM_SOURCE_H_INCLUDED_ - - -typedef struct { - uint32_t hash; - - unsigned value_len:23; - unsigned skip:1; - unsigned name_len:8; - - u_char *value_start; - u_char *name_start; -} nxt_name_value_t; - - -typedef struct { - nxt_list_t *list; - nxt_lvlhsh_t hash; - - uint16_t status; /* 16 bits */ - - nxt_off_t content_length; -} nxt_upstream_header_in_t; - - -typedef nxt_int_t (*nxt_upstream_name_value_handler_t)( - nxt_upstream_source_t *us, nxt_name_value_t *nv); - - -typedef struct { - nxt_upstream_name_value_handler_t handler; - - uint8_t len; - /* - * A name is inlined to test it with one memory access. - * The struct size is aligned to 32 bytes. - */ -#if (NXT_64BIT) - u_char name[23]; -#else - u_char name[27]; -#endif -} nxt_upstream_name_value_t; - - -struct nxt_upstream_source_s { - nxt_upstream_peer_t *peer; - - const nxt_upstream_state_t *state; - - void *protocol_source; - void *data; - nxt_work_queue_t *work_queue; - - nxt_buf_pool_t buffers; - - nxt_lvlhsh_t header_hash; - nxt_stream_source_t *stream; -}; - - -#define NXT_UPSTREAM_NAME_VALUE_MIN_SIZE \ - offsetof(nxt_http_upstream_header_t, name) - -#define nxt_upstream_name_value(s) nxt_length(s), s - - -NXT_EXPORT nxt_int_t nxt_upstream_header_hash_add(nxt_mp_t *mp, - nxt_lvlhsh_t *lh, const nxt_upstream_name_value_t *unv, nxt_uint_t n); -NXT_EXPORT nxt_int_t nxt_upstream_name_value_ignore(nxt_upstream_source_t *us, - nxt_name_value_t *nv); - -NXT_EXPORT extern const nxt_lvlhsh_proto_t nxt_upstream_header_hash_proto; - - -#endif /* _NXT_UPSTREAM_SOURCE_H_INCLUDED_ */ diff --git a/src/perl/nxt_perl_psgi.c b/src/perl/nxt_perl_psgi.c index 5e8d1aee..807d1741 100644 --- a/src/perl/nxt_perl_psgi.c +++ b/src/perl/nxt_perl_psgi.c @@ -267,8 +267,11 @@ XS(XS_NGINX__Unit__Sandbox_cb) XSRETURN_EMPTY; } + pctx = CvXSUBANY(cv).any_ptr; + if (nxt_slow_path(SvOK(ST(0)) == 0 || SvROK(ST(0)) == 0 - || SvTYPE(SvRV(ST(0))) != SVt_PVAV)) + || SvTYPE(SvRV(ST(0))) != SVt_PVAV + || pctx->req == NULL)) { nxt_perl_psgi_cb_request_done(CvXSUBANY(cv).any_ptr, NXT_UNIT_ERROR); @@ -278,8 +281,6 @@ XS(XS_NGINX__Unit__Sandbox_cb) XSRETURN_EMPTY; } - pctx = CvXSUBANY(cv).any_ptr; - rc = nxt_perl_psgi_result_array(PERL_GET_CONTEXT, ST(0), pctx->req); if (nxt_slow_path(rc != NXT_UNIT_OK)) { nxt_perl_psgi_cb_request_done(CvXSUBANY(cv).any_ptr, NXT_UNIT_ERROR); diff --git a/test/conftest.py b/test/conftest.py index 4a1aa7cc..926d83f8 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -107,7 +107,7 @@ def pytest_configure(config): option.current_dir = os.path.abspath( os.path.join(os.path.dirname(__file__), os.pardir) ) - option.test_dir = option.current_dir + '/test' + option.test_dir = f'{option.current_dir}/test' option.architecture = platform.architecture()[0] option.system = platform.system() @@ -148,8 +148,8 @@ def pytest_generate_tests(metafunc): for version in versions: option.generated_tests[ - metafunc.function.__name__ + '[{}]'.format(version) - ] = (type + ' ' + version) + f'{metafunc.function.__name__} [{version}]' + ] = f'{type} {version}' # take available module from option and generate tests for each version @@ -161,18 +161,17 @@ def pytest_generate_tests(metafunc): generate_tests(available_versions) elif prereq_version == 'any': - option.generated_tests[metafunc.function.__name__] = ( - type + ' ' + available_versions[0] - ) + option.generated_tests[ + metafunc.function.__name__ + ] = f'{type} {available_versions[0]}' elif callable(prereq_version): generate_tests(list(filter(prereq_version, available_versions))) else: raise ValueError( - """ -Unexpected prerequisite version "%s" for module "%s" in %s. -'all', 'any' or callable expected.""" - % (str(prereq_version), module, str(cls)) + f''' +Unexpected prerequisite version "{prereq_version}" for module "{module}" in +{cls}. 'all', 'any' or callable expected.''' ) @@ -225,7 +224,7 @@ def pytest_sessionstart(session): check_isolation() check_unix_abstract() - _clear_conf(unit['temp_dir'] + '/control.unit.sock') + _clear_conf(f'{unit["temp_dir"]}/control.unit.sock') unit_stop() @@ -233,6 +232,8 @@ def pytest_sessionstart(session): if option.restart: shutil.rmtree(unit_instance['temp_dir']) + else: + _clear_temp_dir() @pytest.hookimpl(tryfirst=True, hookwrapper=True) @@ -244,7 +245,7 @@ def pytest_runtest_makereport(item, call): # set a report attribute for each phase of a call, which can # be "setup", "call", "teardown" - setattr(item, "rep_" + rep.when, rep) + setattr(item, f'rep_{rep.when}', rep) @pytest.fixture(scope='class', autouse=True) @@ -264,7 +265,7 @@ def check_prerequisites(request): missed.append(module) if missed: - pytest.skip('Unit has no ' + ', '.join(missed) + ' module(s)') + pytest.skip(f'Unit has no {", ".join(missed)} module(s)') # check features @@ -278,7 +279,7 @@ def check_prerequisites(request): missed.append(feature) if missed: - pytest.skip(', '.join(missed) + ' feature(s) not supported') + pytest.skip(f'{", ".join(missed)} feature(s) not supported') @pytest.fixture(autouse=True) @@ -316,33 +317,8 @@ def run(request): # clean temp_dir before the next test if not option.restart: - _clear_conf(unit['temp_dir'] + '/control.unit.sock', log=log) - - if is_findmnt and not waitforunmount(unit['temp_dir'], timeout=600): - exit('Could not unmount some filesystems in tmp dir.') - - for item in os.listdir(unit['temp_dir']): - if item not in [ - 'control.unit.sock', - 'state', - 'unit.pid', - 'unit.log', - ]: - path = os.path.join(unit['temp_dir'], item) - - public_dir(path) - - if os.path.isfile(path) or stat.S_ISSOCK(os.stat(path).st_mode): - os.remove(path) - else: - for attempt in range(10): - try: - shutil.rmtree(path) - break - except OSError as err: - if err.errno != 16: - raise - time.sleep(1) + _clear_conf(f'{unit["temp_dir"]}/control.unit.sock', log=log) + _clear_temp_dir() # check descriptors @@ -374,8 +350,11 @@ def unit_run(state_dir=None): if not option.restart and 'unitd' in unit_instance: return unit_instance - build_dir = option.current_dir + '/build' - unitd = build_dir + '/unitd' + builddir = f'{option.current_dir}/build' + libdir = f'{builddir}/lib' + modulesdir = f'{libdir}/unit/modules' + sbindir = f'{builddir}/sbin' + unitd = f'{sbindir}/unitd' if not os.path.isfile(unitd): exit('Could not find unit') @@ -383,53 +362,55 @@ def unit_run(state_dir=None): temp_dir = tempfile.mkdtemp(prefix='unit-test-') public_dir(temp_dir) - if oct(stat.S_IMODE(os.stat(build_dir).st_mode)) != '0o777': - public_dir(build_dir) + if oct(stat.S_IMODE(os.stat(builddir).st_mode)) != '0o777': + public_dir(builddir) - state = temp_dir + '/state' if state_dir is None else state_dir - if not os.path.isdir(state): - os.mkdir(state) + statedir = f'{temp_dir}/state' if state_dir is None else state_dir + if not os.path.isdir(statedir): + os.mkdir(statedir) + + control_sock = f'{temp_dir}/control.unit.sock' unitd_args = [ unitd, '--no-daemon', - '--modules', - build_dir, - '--state', - state, + '--modulesdir', + modulesdir, + '--statedir', + statedir, '--pid', - temp_dir + '/unit.pid', + f'{temp_dir}/unit.pid', '--log', - temp_dir + '/unit.log', + f'{temp_dir}/unit.log', '--control', - 'unix:' + temp_dir + '/control.unit.sock', - '--tmp', + f'unix:{temp_dir}/control.unit.sock', + '--tmpdir', temp_dir, ] if option.user: unitd_args.extend(['--user', option.user]) - with open(temp_dir + '/unit.log', 'w') as log: + with open(f'{temp_dir}/unit.log', 'w') as log: unit_instance['process'] = subprocess.Popen(unitd_args, stderr=log) Log.temp_dir = temp_dir - if not waitforfiles(temp_dir + '/control.unit.sock'): + if not waitforfiles(control_sock): _print_log() exit('Could not start unit') unit_instance['temp_dir'] = temp_dir - unit_instance['control_sock'] = temp_dir + '/control.unit.sock' + unit_instance['control_sock'] = control_sock unit_instance['unitd'] = unitd option.temp_dir = temp_dir - with open(temp_dir + '/unit.pid', 'r') as f: + with open(f'{temp_dir}/unit.pid', 'r') as f: unit_instance['pid'] = f.read().rstrip() if state_dir is None: - _clear_conf(unit_instance['temp_dir'] + '/control.unit.sock') + _clear_conf(control_sock) _fds_info['main']['fds'] = _count_fds(unit_instance['pid']) @@ -473,7 +454,7 @@ def unit_stop(): try: retcode = p.wait(15) if retcode: - return 'Child process terminated with code ' + str(retcode) + return f'Child process terminated with code {retcode}' except KeyboardInterrupt: p.kill() @@ -518,7 +499,7 @@ def _check_alerts(*, log=None): def _print_log(log=None): path = Log.get_path() - print('Path to unit.log:\n' + path + '\n') + print(f'Path to unit.log:\n{path}\n') if option.print_log: os.set_blocking(sys.stdout.fileno(), True) @@ -542,25 +523,58 @@ def _clear_conf(sock, *, log=None): assert 'success' in resp, 'clear conf' - if 'openssl' not in option.available['modules']: - return + def get(url): + return http.get(url=url, sock_type='unix', addr=sock)['body'] - try: - certs = json.loads( - http.get(url='/certificates', sock_type='unix', addr=sock)['body'] - ).keys() + def delete(url): + return http.delete(url=url, sock_type='unix', addr=sock)['body'] - except json.JSONDecodeError: - pytest.fail('Can\'t parse certificates list.') + if 'openssl' in option.available['modules']: + try: + certs = json.loads(get('/certificates')).keys() - for cert in certs: - resp = http.delete( - url='/certificates/' + cert, - sock_type='unix', - addr=sock, - )['body'] + except json.JSONDecodeError: + pytest.fail("Can't parse certificates list.") - assert 'success' in resp, 'remove certificate' + for cert in certs: + assert 'success' in delete(f'/certificates/{cert}'), 'delete cert' + + if 'njs' in option.available['modules']: + try: + scripts = json.loads(get('/js_modules')).keys() + + except json.JSONDecodeError: + pytest.fail("Can't parse njs modules list.") + + for script in scripts: + assert 'success' in delete(f'/js_modules/{script}'), 'delete script' + +def _clear_temp_dir(): + temp_dir = unit_instance['temp_dir'] + + if is_findmnt and not waitforunmount(temp_dir, timeout=600): + exit('Could not unmount some filesystems in tmpdir ({temp_dir}).') + + for item in os.listdir(temp_dir): + if item not in [ + 'control.unit.sock', + 'state', + 'unit.pid', + 'unit.log', + ]: + path = os.path.join(temp_dir, item) + public_dir(path) + if os.path.isfile(path) or stat.S_ISSOCK(os.stat(path).st_mode): + os.remove(path) + else: + for attempt in range(10): + try: + shutil.rmtree(path) + break + except OSError as err: + if err.errno != 16: + raise + time.sleep(1) def _check_processes(): @@ -595,17 +609,14 @@ def _check_processes(): out = [ l for l in out - if re.search(router_pid + r'\s+' + unit_pid + r'.*unit: router', l) - is None + if re.search(fr'{router_pid}\s+{unit_pid}.*unit: router', l) is None ] assert len(out) == 1, 'one router' out = [ l for l in out - if re.search( - controller_pid + r'\s+' + unit_pid + r'.*unit: controller', l - ) + if re.search(fr'{controller_pid}\s+{unit_pid}.*unit: controller', l) is None ] assert len(out) == 0, 'one controller' @@ -646,18 +657,16 @@ def _check_fds(*, log=None): ps['fds'] += fds_diff if not option.restart: - assert ps['pid'] == ps_pid, 'same pid %s' % name + assert ps['pid'] == ps_pid, f'same pid {name}' - assert fds_diff <= option.fds_threshold, ( - 'descriptors leak %s' % name - ) + assert fds_diff <= option.fds_threshold, f'descriptors leak {name}' else: ps['fds'] = _count_fds(ps['pid']) def _count_fds(pid): - procfile = '/proc/%s/fd' % pid + procfile = f'/proc/{pid}/fd' if os.path.isdir(procfile): return len(os.listdir(procfile)) @@ -712,14 +721,12 @@ def stop_processes(): def pid_by_name(name): output = subprocess.check_output(['ps', 'ax', '-O', 'ppid']).decode() - m = re.search( - r'\s*(\d+)\s*' + str(unit_instance['pid']) + r'.*' + name, output - ) + m = re.search(fr'\s*(\d+)\s*{unit_instance["pid"]}.*{name}', output) return None if m is None else m.group(1) def find_proc(name, ps_output): - return re.findall(str(unit_instance['pid']) + r'.*' + name, ps_output) + return re.findall(f'{unit_instance["pid"]}.*{name}', ps_output) @pytest.fixture() @@ -762,7 +769,7 @@ def unit_pid(request): def pytest_sessionfinish(session): if not option.restart and option.save_log: - print('Path to unit.log:\n' + Log.get_path() + '\n') + print(f'Path to unit.log:\n{Log.get_path()}\n') option.restart = True diff --git a/test/njs/global_this/script.js b/test/njs/global_this/script.js new file mode 100644 index 00000000..cf9bc078 --- /dev/null +++ b/test/njs/global_this/script.js @@ -0,0 +1,3 @@ +export default { + "str" : function () {return typeof globalThis.njs.version} +} diff --git a/test/njs/import_from/script.js b/test/njs/import_from/script.js new file mode 100644 index 00000000..99d1b80e --- /dev/null +++ b/test/njs/import_from/script.js @@ -0,0 +1,5 @@ +import cr from 'crypto'; + +export default { + "num" : function () {return typeof cr.createHash('md5').digest().length} +} diff --git a/test/njs/invalid/script.js b/test/njs/invalid/script.js new file mode 100644 index 00000000..a9d79797 --- /dev/null +++ b/test/njs/invalid/script.js @@ -0,0 +1,3 @@ +export default { + "route": function() {blah 'next'} +} diff --git a/test/njs/next/script.js b/test/njs/next/script.js new file mode 100644 index 00000000..86e49e82 --- /dev/null +++ b/test/njs/next/script.js @@ -0,0 +1,3 @@ +export default { + "route": function() {return 'next'} +} diff --git a/test/perl/streaming_body_multiple_responses/psgi.pl b/test/perl/streaming_body_multiple_responses/psgi.pl new file mode 100644 index 00000000..ffd026bd --- /dev/null +++ b/test/perl/streaming_body_multiple_responses/psgi.pl @@ -0,0 +1,11 @@ +my $counter = 2; + +my $app = sub { + my $env = shift; + + return sub { + my $responder = shift; + $responder->([200, ['Content-Type'=>'text/plain'], [$counter++]]); + $responder->([200, ['Content-Type'=>'text/plain'], [$counter++]]); + }; +}; diff --git a/test/python/environment/wsgi.py b/test/python/environment/wsgi.py index d1564f29..91e0ba49 100644 --- a/test/python/environment/wsgi.py +++ b/test/python/environment/wsgi.py @@ -2,13 +2,10 @@ import os def application(env, start_response): - body = '' vars = env.get('HTTP_X_VARIABLES').split(',') - for var in vars: - if var in os.environ: - body += str(os.environ[var]) + ',' - + body = ','.join([str(os.environ[var]) for var in vars if var in os.environ]) body = body.encode() + start_response('200', [('Content-Length', str(len(body)))]) return body diff --git a/test/python/input_readline_size/wsgi.py b/test/python/input_readline_size/wsgi.py index 36cf07b0..bde8c0d4 100644 --- a/test/python/input_readline_size/wsgi.py +++ b/test/python/input_readline_size/wsgi.py @@ -9,7 +9,7 @@ def application(environ, start_response): body.append(l) if len(l) > 9: - body.append(b'len(l) > 9: ' + l) + body.append(f'len(l) > 9: {l}'.encode()) break start_response('200', [('X-Lines-Count', str(len(body)))]) diff --git a/test/python/lifespan/empty/asgi.py b/test/python/lifespan/empty/asgi.py index 8ceecc2f..27395a28 100644 --- a/test/python/lifespan/empty/asgi.py +++ b/test/python/lifespan/empty/asgi.py @@ -3,17 +3,17 @@ import os async def handler(prefix, scope, receive, send): if scope['type'] == 'lifespan': - with open(prefix + 'version', 'w+') as f: + with open(f'{prefix}version', 'w+') as f: f.write( - scope['asgi']['version'] + ' ' + scope['asgi']['spec_version'] + f"{scope['asgi']['version']} {scope['asgi']['spec_version']}" ) while True: message = await receive() if message['type'] == 'lifespan.startup': - os.remove(prefix + 'startup') + os.remove(f'{prefix}startup') await send({'type': 'lifespan.startup.complete'}) elif message['type'] == 'lifespan.shutdown': - os.remove(prefix + 'shutdown') + os.remove(f'{prefix}shutdown') await send({'type': 'lifespan.shutdown.complete'}) return @@ -22,7 +22,9 @@ async def handler(prefix, scope, receive, send): { 'type': 'http.response.start', 'status': 204, - 'headers': [(b'content-length', b'0'),], + 'headers': [ + (b'content-length', b'0'), + ], } ) diff --git a/test/python/targets/wsgi.py b/test/python/targets/wsgi.py index 3f3d4b27..30a50efd 100644 --- a/test/python/targets/wsgi.py +++ b/test/python/targets/wsgi.py @@ -9,9 +9,6 @@ def wsgi_target_b(env, start_response): def wsgi_target_prefix(env, start_response): - data = u'%s %s' % ( - env.get('SCRIPT_NAME', 'No Script Name'), - env['PATH_INFO'], - ) - start_response('200', [('Content-Length', '%d' % len(data))]) + data = f"{env.get('SCRIPT_NAME', 'No Script Name')} {env['PATH_INFO']}" + start_response('200', [('Content-Length', f'{data}')]) return [data.encode('utf-8')] diff --git a/test/python/threading/asgi.py b/test/python/threading/asgi.py index fed6fcce..aa9b76cf 100644 --- a/test/python/threading/asgi.py +++ b/test/python/threading/asgi.py @@ -11,9 +11,7 @@ class Foo(threading.Thread): threading.Thread.__init__(self) def log_index(self, index): - sys.stderr.write( - "(" + str(index) + ") Thread: " + str(self.__x) + "\n" - ) + sys.stderr.write(f'({index}) Thread: {self.__x}\n') sys.stderr.flush() def run(self): diff --git a/test/python/threading/wsgi.py b/test/python/threading/wsgi.py index 48a73afd..afba674b 100644 --- a/test/python/threading/wsgi.py +++ b/test/python/threading/wsgi.py @@ -11,9 +11,7 @@ class Foo(threading.Thread): threading.Thread.__init__(self) def log_index(self, index): - sys.stderr.write( - "(" + str(index) + ") Thread: " + str(self.__x) + "\n" - ) + sys.stderr.write(f'({index}) Thread: {self.__x}\n') sys.stderr.flush() def run(self): diff --git a/test/python/unicode/wsgi.py b/test/python/unicode/wsgi.py index 40043af9..f2f85f5d 100644 --- a/test/python/unicode/wsgi.py +++ b/test/python/unicode/wsgi.py @@ -1,7 +1,7 @@ def application(environ, start_response): temp_dir = environ.get('HTTP_TEMP_DIR') - with open(temp_dir + '/tempfile', 'w') as f: + with open(f'{temp_dir}/tempfile', 'w') as f: f.write('\u26a0\ufe0f') start_response('200', [('Content-Length', '0')]) diff --git a/test/test_access_log.py b/test/test_access_log.py index a072858b..c29638a3 100644 --- a/test/test_access_log.py +++ b/test/test_access_log.py @@ -12,13 +12,13 @@ class TestAccessLog(TestApplicationPython): super().load(script) assert 'success' in self.conf( - '"' + option.temp_dir + '/access.log"', 'access_log' + f'"{option.temp_dir}/access.log"', 'access_log' ), 'access_log configure' def set_format(self, format): assert 'success' in self.conf( { - 'path': option.temp_dir + '/access.log', + 'path': f'{option.temp_dir}/access.log', 'format': format, }, 'access_log', @@ -103,13 +103,13 @@ Connection: close is not None ), 'ipv6' - def test_access_log_unix(self): + def test_access_log_unix(self, temp_dir): self.load('empty') - addr = option.temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( - {"unix:" + addr: {"pass": "applications/empty"}}, 'listeners' + {f'unix:{addr}': {"pass": "applications/empty"}}, 'listeners' ) self.get(sock_type='unix', addr=addr) @@ -174,7 +174,7 @@ Connection: close time.sleep(1) assert ( - self.wait_for_record(r'"GE" 400 0 "-" "-"') is not None + self.wait_for_record(r'"-" 400 0 "-" "-"') is not None ), 'partial' def test_access_log_partial_2(self): @@ -185,7 +185,7 @@ Connection: close self.http(b"""GET /\n""", raw=True) assert ( - self.wait_for_record(r'"GET /" 400 \d+ "-" "-"') is not None + self.wait_for_record(r'"-" 400 \d+ "-" "-"') is not None ), 'partial 2' def test_access_log_partial_3(self): @@ -198,7 +198,7 @@ Connection: close time.sleep(1) assert ( - self.wait_for_record(r'"GET /" 400 0 "-" "-"') is not None + self.wait_for_record(r'"-" 400 0 "-" "-"') is not None ), 'partial 3' def test_access_log_partial_4(self): @@ -211,7 +211,7 @@ Connection: close time.sleep(1) assert ( - self.wait_for_record(r'"GET / HTTP/1.1" 400 0 "-" "-"') is not None + self.wait_for_record(r'"-" 400 0 "-" "-"') is not None ), 'partial 4' @pytest.mark.skip('not yet') @@ -248,14 +248,12 @@ Connection: close assert self.search_in_log(r'/delete', 'access.log') is None, 'delete' - def test_access_log_change(self): + def test_access_log_change(self, temp_dir): self.load('empty') self.get() - assert 'success' in self.conf( - '"' + option.temp_dir + '/new.log"', 'access_log' - ) + assert 'success' in self.conf(f'"{temp_dir}/new.log"', 'access_log') self.get() @@ -286,19 +284,20 @@ Connection: close body = '0123456789' * 50 self.post(url='/bbs', body=body, read_timeout=1) assert ( - self.wait_for_record(r'^\/bbs ' + str(len(body)) + r'$') is not None + self.wait_for_record(fr'^\/bbs {len(body)}$') is not None ), '$body_bytes_sent' - def test_access_log_incorrect(self, skip_alert): + def test_access_log_incorrect(self, temp_dir, skip_alert): skip_alert(r'failed to apply new conf') assert 'error' in self.conf( - option.temp_dir + '/blah/access.log' 'access_log/path', + f'{option.temp_dir}/blah/access.log', + 'access_log/path', ), 'access_log path incorrect' assert 'error' in self.conf( { - 'path': option.temp_dir + '/access.log', + 'path': f'{temp_dir}/access.log', 'format': '$remote_add', }, 'access_log', diff --git a/test/test_asgi_application.py b/test/test_asgi_application.py index 121a2fbc..5ce82cb2 100644 --- a/test/test_asgi_application.py +++ b/test/test_asgi_application.py @@ -20,17 +20,16 @@ class TestASGIApplication(TestApplicationPython): body = 'Test body string.' resp = self.http( - b"""POST / HTTP/1.1 + f"""POST / HTTP/1.1 Host: localhost -Content-Length: %d +Content-Length: {len(body)} Custom-Header: blah Custom-hEader: Blah Content-Type: text/html Connection: close custom-header: BLAH -%s""" - % (len(body), body.encode()), +{body}""".encode(), raw=True, ) @@ -63,9 +62,9 @@ custom-header: BLAH def test_asgi_application_unix(self, temp_dir): self.load('empty') - addr = temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( - {"unix:" + addr: {"pass": "applications/empty"}}, 'listeners' + {f"unix:{addr}": {"pass": "applications/empty"}}, 'listeners' ) assert self.get(sock_type='unix', addr=addr)['status'] == 200 @@ -83,7 +82,7 @@ custom-header: BLAH self.load('prefix', prefix='/api/rest') def set_prefix(prefix): - self.conf('"' + prefix + '"', 'applications/prefix/prefix') + self.conf(f'"{prefix}"', 'applications/prefix/prefix') def check_prefix(url, prefix): resp = self.get(url=url) @@ -190,7 +189,7 @@ custom-header: BLAH max_body_size = 12 * 1024 * 1024 assert 'success' in self.conf( - '{"http":{"max_body_size": ' + str(max_body_size) + ' }}', + f'{{"http":{{"max_body_size": {max_body_size} }}}}', 'settings', ) diff --git a/test/test_asgi_application_unix_abstract.py b/test/test_asgi_application_unix_abstract.py index c4ec812f..2ca7839f 100644 --- a/test/test_asgi_application_unix_abstract.py +++ b/test/test_asgi_application_unix_abstract.py @@ -16,7 +16,7 @@ class TestASGIApplicationUnixAbstract(TestApplicationPython): addr = '\0sock' assert 'success' in self.conf( - {"unix:@" + addr[1:]: {"pass": "applications/empty"}}, + {f"unix:@{addr[1:]}": {"pass": "applications/empty"}}, 'listeners', ) diff --git a/test/test_asgi_lifespan.py b/test/test_asgi_lifespan.py index e295f7fa..84e9fea4 100644 --- a/test/test_asgi_lifespan.py +++ b/test/test_asgi_lifespan.py @@ -15,25 +15,25 @@ class TestASGILifespan(TestApplicationPython): load_module = 'asgi' def setup_cookies(self, prefix): - base_dir = option.test_dir + '/python/lifespan/empty' + base_dir = f'{option.test_dir}/python/lifespan/empty' os.chmod(base_dir, 0o777) for name in ['startup', 'shutdown', 'version']: - path = option.test_dir + '/python/lifespan/empty/' + prefix + name + path = f'{option.test_dir}/python/lifespan/empty/{prefix}{name}' open(path, 'a').close() os.chmod(path, 0o777) def assert_cookies(self, prefix): for name in ['startup', 'shutdown']: - path = option.test_dir + '/python/lifespan/empty/' + prefix + name + path = f'{option.test_dir}/python/lifespan/empty/{prefix}{name}' exists = os.path.isfile(path) if exists: os.remove(path) assert not exists, name - path = option.test_dir + '/python/lifespan/empty/' + prefix + 'version' + path = f'{option.test_dir}/python/lifespan/empty/{prefix}version' with open(path, 'r') as f: version = f.read() @@ -54,6 +54,8 @@ class TestASGILifespan(TestApplicationPython): self.assert_cookies('') def test_asgi_lifespan_targets(self): + path = f'{option.test_dir}/python/lifespan/empty' + assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, @@ -71,9 +73,8 @@ class TestASGILifespan(TestApplicationPython): "targets": { "type": self.get_application_type(), "processes": {"spare": 0}, - "working_directory": option.test_dir - + "/python/lifespan/empty", - "path": option.test_dir + '/python/lifespan/empty', + "working_directory": path, + "path": path, "targets": { "1": {"module": "asgi", "callable": "application"}, "2": { diff --git a/test/test_asgi_targets.py b/test/test_asgi_targets.py index 84d7b3b0..5afc7079 100644 --- a/test/test_asgi_targets.py +++ b/test/test_asgi_targets.py @@ -14,6 +14,8 @@ class TestASGITargets(TestApplicationPython): @pytest.fixture(autouse=True) def setup_method_fixture(self): + path = f'{option.test_dir}/python/targets/' + assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, @@ -31,9 +33,8 @@ class TestASGITargets(TestApplicationPython): "targets": { "type": self.get_application_type(), "processes": {"spare": 0}, - "working_directory": option.test_dir - + "/python/targets/", - "path": option.test_dir + '/python/targets/', + "working_directory": path, + "path": path, "protocol": "asgi", "targets": { "1": { diff --git a/test/test_asgi_websockets.py b/test/test_asgi_websockets.py index 975be90a..b15bee43 100644 --- a/test/test_asgi_websockets.py +++ b/test/test_asgi_websockets.py @@ -151,7 +151,7 @@ class TestASGIWebsockets(TestApplicationPython): frame = self.ws.frame_read(sock) - assert message + ' ' + message == frame['data'].decode( + assert f'{message} {message}' == frame['data'].decode( 'utf-8' ), 'mirror framing' @@ -527,21 +527,21 @@ class TestASGIWebsockets(TestApplicationPython): _, sock, _ = self.ws.upgrade() for i in range(0, 10): - self.ws.frame_write(sock, self.ws.OP_PING, 'payload-%d' % i) + self.ws.frame_write(sock, self.ws.OP_PING, f'payload-{i}') for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') # 2_11 for i in range(0, 10): opcode = self.ws.OP_PING - self.ws.frame_write(sock, opcode, 'payload-%d' % i, chopsize=1) + self.ws.frame_write(sock, opcode, f'payload-{i}', chopsize=1) for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') self.close_connection(sock) diff --git a/test/test_client_ip.py b/test/test_client_ip.py index 50aa6afc..6520d5e2 100644 --- a/test/test_client_ip.py +++ b/test/test_client_ip.py @@ -16,9 +16,7 @@ class TestClientIP(TestApplicationPython): "client_ip": options, "pass": "applications/client_ip", }, - "unix:" - + option.temp_dir - + "/sock": { + f"unix:{option.temp_dir}/sock": { "client_ip": options, "pass": "applications/client_ip", }, @@ -30,7 +28,7 @@ class TestClientIP(TestApplicationPython): address = { 'ipv4': ('127.0.0.1', 7081), 'ipv6': ('::1', 7082), - 'unix': (option.temp_dir + '/sock', None), + 'unix': (f'{option.temp_dir}/sock', None), } (addr, port) = address[sock_type] diff --git a/test/test_configuration.py b/test/test_configuration.py index 9c27222c..e3ddc891 100644 --- a/test/test_configuration.py +++ b/test/test_configuration.py @@ -283,7 +283,7 @@ class TestConfiguration(TestControl): assert 'success' in self.conf( { - "listeners": {"*:7080": {"pass": "applications/" + name}}, + "listeners": {"*:7080": {"pass": f"applications/{name}"}}, "applications": { name: { "type": "python", @@ -301,8 +301,7 @@ class TestConfiguration(TestControl): conf = { "applications": { - "app-" - + str(a): { + f"app-{a}": { "type": "python", "processes": {"spare": 0}, "path": "/app", @@ -311,7 +310,7 @@ class TestConfiguration(TestControl): for a in range(apps) }, "listeners": { - "*:" + str(7000 + a): {"pass": "applications/app-" + str(a)} + f"*:{(7000 + a)}": {"pass": f"applications/app-{a}"} for a in range(apps) }, } @@ -407,8 +406,7 @@ class TestConfiguration(TestControl): def test_json_application_many2(self): conf = { "applications": { - "app-" - + str(a): { + f"app-{a}": { "type": "python", "processes": {"spare": 0}, "path": "/app", diff --git a/test/test_go_application.py b/test/test_go_application.py index a746c6f4..9034d5aa 100644 --- a/test/test_go_application.py +++ b/test/test_go_application.py @@ -139,15 +139,15 @@ class TestGoApplication(TestApplicationGo): self.load('command_line_arguments') arg1 = '--cc=gcc-7.2.0' - arg2 = '--cc-opt=\'-O0 -DNXT_DEBUG_MEMORY=1 -fsanitize=address\'' + arg2 = "--cc-opt='-O0 -DNXT_DEBUG_MEMORY=1 -fsanitize=address'" arg3 = '--debug' assert 'success' in self.conf( - '["' + arg1 + '", "' + arg2 + '", "' + arg3 + '"]', + f'["{arg1}", "{arg2}", "{arg3}"]', 'applications/command_line_arguments/arguments', ) - assert self.get()['body'] == arg1 + ',' + arg2 + ',' + arg3, 'arguments' + assert self.get()['body'] == f'{arg1},{arg2},{arg3}', 'arguments' def test_go_application_command_line_arguments_change(self): self.load('command_line_arguments') diff --git a/test/test_go_isolation.py b/test/test_go_isolation.py index c3f92679..f063f987 100644 --- a/test/test_go_isolation.py +++ b/test/test_go_isolation.py @@ -37,7 +37,7 @@ class TestGoIsolation(TestApplicationGo): for ns, ns_value in option.available['features']['isolation'].items(): if ns.upper() in obj['NS']: - assert obj['NS'][ns.upper()] == ns_value, '%s match' % ns + assert obj['NS'][ns.upper()] == ns_value, f'{ns} match' def test_isolation_unpriv_user(self, is_su): if not self.isolation_key('unprivileged_userns_clone'): @@ -60,7 +60,7 @@ class TestGoIsolation(TestApplicationGo): # unprivileged unit map itself to nobody in the container by default assert obj['UID'] == nobody_uid, 'uid of nobody' - assert obj['GID'] == nogroup_gid, 'gid of %s' % nogroup + assert obj['GID'] == nogroup_gid, f'gid of {nogroup}' self.load( 'ns_inspect', @@ -200,7 +200,7 @@ class TestGoIsolation(TestApplicationGo): assert ( obj['NS'][ns.upper()] == option.available['features']['isolation'][ns] - ), ('%s match' % ns) + ), f'{ns} match' assert obj['NS']['MNT'] != getns('mnt'), 'mnt set' assert obj['NS']['USER'] != getns('user'), 'user set' @@ -260,7 +260,7 @@ class TestGoIsolation(TestApplicationGo): assert ( obj['NS'][ns.upper()] == option.available['features']['isolation'][ns] - ), ('%s match' % ns) + ), f'{ns} match' def test_go_isolation_rootfs_container(self, is_su, temp_dir): if not is_su: diff --git a/test/test_http_header.py b/test/test_http_header.py index 6773c44f..cae5e9b8 100644 --- a/test/test_http_header.py +++ b/test/test_http_header.py @@ -107,7 +107,7 @@ class TestHTTPHeader(TestApplicationPython): resp = self.get( headers={ 'Host': 'localhost', - 'Custom-Header': r'(),/:;<=>?@[\]{}\t !#$%&\'*+-.^_`|~', + 'Custom-Header': r"(),/:;<=>?@[\]{}\t !#$%&'*+-.^_`|~", 'Connection': 'close', } ) @@ -115,7 +115,7 @@ class TestHTTPHeader(TestApplicationPython): assert resp['status'] == 200, 'value chars status' assert ( resp['headers']['Custom-Header'] - == r'(),/:;<=>?@[\]{}\t !#$%&\'*+-.^_`|~' + == r"(),/:;<=>?@[\]{}\t !#$%&'*+-.^_`|~" ), 'value chars custom header' def test_http_header_value_chars_edge(self): diff --git a/test/test_java_application.py b/test/test_java_application.py index b825d925..6ff556a8 100644 --- a/test/test_java_application.py +++ b/test/test_java_application.py @@ -24,9 +24,9 @@ class TestJavaApplication(TestApplicationJava): "app": { "type": self.get_application_type(), "processes": 1, - "working_directory": option.test_dir + "/java/empty", - "webapp": temp_dir + "/java", - "unit_jars": temp_dir + "/no_such_dir", + "working_directory": f"{option.test_dir}/java/empty", + "webapp": f"{temp_dir}/java", + "unit_jars": f"{temp_dir}/no_such_dir", } }, } @@ -36,7 +36,7 @@ class TestJavaApplication(TestApplicationJava): self.load('empty_war') assert 'success' in self.conf( - '"' + temp_dir + '/java/empty.war"', + f'"{temp_dir}/java/empty.war"', '/config/applications/empty_war/webapp', ), 'configure war' @@ -118,7 +118,7 @@ class TestJavaApplication(TestApplicationJava): headers = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', }, url='/?var1=val2', @@ -157,7 +157,7 @@ class TestJavaApplication(TestApplicationJava): resp = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', } ) @@ -171,7 +171,7 @@ class TestJavaApplication(TestApplicationJava): resp = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', } ) @@ -183,7 +183,7 @@ class TestJavaApplication(TestApplicationJava): resp = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', } ) @@ -207,7 +207,7 @@ class TestJavaApplication(TestApplicationJava): resp = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', } ) @@ -223,7 +223,7 @@ class TestJavaApplication(TestApplicationJava): resp = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', } ) @@ -244,7 +244,7 @@ class TestJavaApplication(TestApplicationJava): headers = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', }, url='/?var1=val2', @@ -256,7 +256,7 @@ class TestJavaApplication(TestApplicationJava): headers = self.get( headers={ 'Host': 'localhost', - 'Cookie': 'JSESSIONID=' + session_id, + 'Cookie': f'JSESSIONID={session_id}', 'Connection': 'close', }, url='/', @@ -770,7 +770,7 @@ class TestJavaApplication(TestApplicationJava): assert headers['X-Path-Info'] == '/test', 'the rest of the path' assert ( headers['X-Path-Translated'] - == headers['X-Real-Path'] + headers['X-Path-Info'] + == f"{headers['X-Real-Path']}{headers['X-Path-Info']}" ), 'translated path is the app root + path info' assert ( headers['X-Resource-Paths'].endswith('/WEB-INF/, /index.html]') @@ -956,7 +956,7 @@ class TestJavaApplication(TestApplicationJava): self.load('multipart') reldst = '/uploads' - fulldst = temp_dir + reldst + fulldst = f'{temp_dir}{reldst}' os.mkdir(fulldst) public_dir(fulldst) @@ -989,7 +989,7 @@ class TestJavaApplication(TestApplicationJava): assert re.search(r'sample\.txt created', resp['body']), 'multipart body' assert ( self.search_in_log( - r'^Data from sample file$', name=reldst + '/sample.txt' + r'^Data from sample file$', name=f'{reldst}/sample.txt' ) is not None ), 'file created' diff --git a/test/test_java_isolation_rootfs.py b/test/test_java_isolation_rootfs.py index 3c6a45a3..28668997 100644 --- a/test/test_java_isolation_rootfs.py +++ b/test/test_java_isolation_rootfs.py @@ -13,17 +13,17 @@ class TestJavaIsolationRootfs(TestApplicationJava): if not is_su: pytest.skip('require root') - os.makedirs(option.temp_dir + '/jars') - os.makedirs(option.temp_dir + '/tmp') - os.chmod(option.temp_dir + '/tmp', 0o777) + os.makedirs(f'{option.temp_dir}/jars') + os.makedirs(f'{option.temp_dir}/tmp') + os.chmod(f'{option.temp_dir}/tmp', 0o777) try: subprocess.run( [ "mount", "--bind", - option.current_dir + "/build", - option.temp_dir + "/jars", + f'{option.current_dir}/build', + f'{option.temp_dir}/jars', ], stderr=subprocess.STDOUT, ) @@ -32,7 +32,7 @@ class TestJavaIsolationRootfs(TestApplicationJava): raise except subprocess.CalledProcessError: - pytest.fail('Can\'t run mount process.') + pytest.fail("Can't run mount process.") def teardown_method(self, is_su): if not is_su: @@ -40,7 +40,7 @@ class TestJavaIsolationRootfs(TestApplicationJava): try: subprocess.run( - ["umount", "--lazy", option.temp_dir + "/jars"], + ["umount", "--lazy", f"{option.temp_dir}/jars"], stderr=subprocess.STDOUT, ) @@ -48,7 +48,7 @@ class TestJavaIsolationRootfs(TestApplicationJava): raise except subprocess.CalledProcessError: - pytest.fail('Can\'t run umount process.') + pytest.fail("Can't run umount process.") def test_java_isolation_rootfs_chroot_war(self, is_su, temp_dir): if not is_su: diff --git a/test/test_java_websockets.py b/test/test_java_websockets.py index 362c8619..8de45a06 100644 --- a/test/test_java_websockets.py +++ b/test/test_java_websockets.py @@ -109,7 +109,7 @@ class TestJavaWebsockets(TestApplicationJava): frame = self.ws.frame_read(sock) - assert message + ' ' + message == frame['data'].decode( + assert f'{message} {message}' == frame['data'].decode( 'utf-8' ), 'mirror framing' @@ -454,21 +454,21 @@ class TestJavaWebsockets(TestApplicationJava): _, sock, _ = self.ws.upgrade() for i in range(0, 10): - self.ws.frame_write(sock, self.ws.OP_PING, 'payload-%d' % i) + self.ws.frame_write(sock, self.ws.OP_PING, f'payload-{i}') for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') # 2_11 for i in range(0, 10): opcode = self.ws.OP_PING - self.ws.frame_write(sock, opcode, 'payload-%d' % i, chopsize=1) + self.ws.frame_write(sock, opcode, f'payload-{i}', chopsize=1) for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') self.close_connection(sock) diff --git a/test/test_njs.py b/test/test_njs.py index 2cbded5b..a7261290 100644 --- a/test/test_njs.py +++ b/test/test_njs.py @@ -2,75 +2,77 @@ import os from unit.applications.proto import TestApplicationProto from unit.option import option +from unit.utils import waitforfiles class TestNJS(TestApplicationProto): prerequisites = {'modules': {'njs': 'any'}} def setup_method(self): - os.makedirs(option.temp_dir + '/assets') - open(option.temp_dir + '/assets/index.html', 'a') - open(option.temp_dir + '/assets/localhost', 'a') - open(option.temp_dir + '/assets/`string`', 'a') - open(option.temp_dir + '/assets/`backtick', 'a') - open(option.temp_dir + '/assets/l1\nl2', 'a') - open(option.temp_dir + '/assets/127.0.0.1', 'a') - assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, "routes": [ - {"action": {"share": option.temp_dir + "/assets$uri"}} + {"action": {"share": f"{option.temp_dir}/assets$uri"}} ], } ) + def create_files(self, *files): + assets_dir = f'{option.temp_dir}/assets/' + os.makedirs(assets_dir) + + [open(assets_dir + f, 'a') for f in files] + waitforfiles(*[assets_dir + f for f in files]) + def set_share(self, share): assert 'success' in self.conf(share, 'routes/0/action/share') - def test_njs_template_string(self, temp_dir): - self.set_share('"`' + temp_dir + '/assets/index.html`"') - assert self.get()['status'] == 200, 'string' + def check_expression(self, expression, url='/'): + self.set_share(f'"`{option.temp_dir}/assets{expression}`"') + assert self.get(url=url)['status'] == 200 - self.set_share('"' + temp_dir + '/assets/`string`"') - assert self.get()['status'] == 200, 'string 2' + def test_njs_template_string(self, temp_dir): + self.create_files('str', '`string`', '`backtick', 'l1\nl2') - self.set_share('"`' + temp_dir + '/assets/\\\\`backtick`"') - assert self.get()['status'] == 200, 'escape' + self.check_expression('/str') + self.check_expression('/\\\\`backtick') + self.check_expression('/l1\\nl2') - self.set_share('"`' + temp_dir + '/assets/l1\\nl2`"') - assert self.get()['status'] == 200, 'multiline' + self.set_share(f'"{temp_dir}/assets/`string`"') + assert self.get()['status'] == 200 def test_njs_template_expression(self, temp_dir): - def check_expression(expression): - self.set_share(expression) - assert self.get()['status'] == 200 + self.create_files('str', 'localhost') - check_expression('"`' + temp_dir + '/assets${uri}`"') - check_expression('"`' + temp_dir + '/assets${uri}${host}`"') - check_expression('"`' + temp_dir + '/assets${uri + host}`"') - check_expression('"`' + temp_dir + '/assets${uri + `${host}`}`"') + self.check_expression('${uri}', '/str') + self.check_expression('${uri}${host}') + self.check_expression('${uri + host}') + self.check_expression('${uri + `${host}`}') - def test_njs_variables(self, temp_dir): - self.set_share('"`' + temp_dir + '/assets/${host}`"') - assert self.get()['status'] == 200, 'host' + def test_njs_iteration(self, temp_dir): + self.create_files('Connection,Host', 'close,localhost') - self.set_share('"`' + temp_dir + '/assets/${remoteAddr}`"') - assert self.get()['status'] == 200, 'remoteAddr' + self.check_expression('/${Object.keys(headers).sort().join()}') + self.check_expression('/${Object.values(headers).sort().join()}') + + def test_njs_variables(self, temp_dir): + self.create_files('str', 'localhost', '127.0.0.1') - self.set_share('"`' + temp_dir + '/assets/${headers.Host}`"') - assert self.get()['status'] == 200, 'headers' + self.check_expression('/${host}') + self.check_expression('/${remoteAddr}') + self.check_expression('/${headers.Host}') - self.set_share('"`' + temp_dir + '/assets/${cookies.foo}`"') + self.set_share(f'"`{temp_dir}/assets/${{cookies.foo}}`"') assert ( - self.get( - headers={'Cookie': 'foo=localhost', 'Connection': 'close'} - )['status'] + self.get(headers={'Cookie': 'foo=str', 'Connection': 'close'})[ + 'status' + ] == 200 ), 'cookies' - self.set_share('"`' + temp_dir + '/assets/${args.foo}`"') - assert self.get(url='/?foo=localhost')['status'] == 200, 'args' + self.set_share(f'"`{temp_dir}/assets/${{args.foo}}`"') + assert self.get(url='/?foo=str')['status'] == 200, 'args' def test_njs_invalid(self, temp_dir, skip_alert): skip_alert(r'js exception:') diff --git a/test/test_njs_modules.py b/test/test_njs_modules.py new file mode 100644 index 00000000..ce592fe4 --- /dev/null +++ b/test/test_njs_modules.py @@ -0,0 +1,99 @@ +from unit.applications.proto import TestApplicationProto +from unit.option import option + + +class TestNJSModules(TestApplicationProto): + prerequisites = {'modules': {'njs': 'any'}} + + def njs_script_load(self, module, name=None, expect='success'): + if name is None: + name = module + + with open(f'{option.test_dir}/njs/{module}/script.js', 'rb') as s: + assert expect in self.conf(s.read(), f'/js_modules/{name}') + + def test_njs_modules(self): + self.njs_script_load('next') + + assert 'export' in self.conf_get('/js_modules/next') + assert 'error' in self.conf_post('"blah"', '/js_modules/next') + + assert 'success' in self.conf( + { + "settings": {"js_module": "next"}, + "listeners": {"*:7080": {"pass": "routes/first"}}, + "routes": { + "first": [{"action": {"pass": "`routes/${next.route()}`"}}], + "next": [{"action": {"return": 200}}], + }, + } + ) + assert self.get()['status'] == 200, 'string' + + assert 'success' in self.conf({"js_module": ["next"]}, 'settings') + assert self.get()['status'] == 200, 'array' + + # add one more value to array + + assert len(self.conf_get('/js_modules').keys()) == 1 + + self.njs_script_load('next', 'next_2') + + assert len(self.conf_get('/js_modules').keys()) == 2 + + assert 'success' in self.conf_post('"next_2"', 'settings/js_module') + assert self.get()['status'] == 200, 'array len 2' + + assert 'success' in self.conf( + '"`routes/${next_2.route()}`"', 'routes/first/0/action/pass' + ) + assert self.get()['status'] == 200, 'array new' + + # can't update exsisting script + + self.njs_script_load('global_this', 'next', expect='error') + + # delete modules + + assert 'error' in self.conf_delete('/js_modules/next_2') + assert 'success' in self.conf_delete('settings/js_module') + assert 'success' in self.conf_delete('/js_modules/next_2') + + def test_njs_modules_import(self): + self.njs_script_load('import_from') + + assert 'success' in self.conf( + { + "settings": {"js_module": "import_from"}, + "listeners": {"*:7080": {"pass": "routes/first"}}, + "routes": { + "first": [ + {"action": {"pass": "`routes/${import_from.num()}`"}} + ], + "number": [{"action": {"return": 200}}], + }, + } + ) + assert self.get()['status'] == 200 + + def test_njs_modules_this(self): + self.njs_script_load('global_this') + + assert 'success' in self.conf( + { + "settings": {"js_module": "global_this"}, + "listeners": {"*:7080": {"pass": "routes/first"}}, + "routes": { + "first": [ + {"action": {"pass": "`routes/${global_this.str()}`"}} + ], + "string": [{"action": {"return": 200}}], + }, + } + ) + assert self.get()['status'] == 200 + + def test_njs_modules_invalid(self, skip_alert): + skip_alert(r'.*JS compile module.*failed.*') + + self.njs_script_load('invalid', expect='error') diff --git a/test/test_node_application.py b/test/test_node_application.py index c26c72d0..719afae8 100644 --- a/test/test_node_application.py +++ b/test/test_node_application.py @@ -146,7 +146,7 @@ class TestNodeApplication(TestApplicationNode): self.load('write_callback') assert self.get()['body'] == 'helloworld', 'write callback order' - assert waitforfiles(temp_dir + '/node/callback'), 'write callback' + assert waitforfiles(f'{temp_dir}/node/callback'), 'write callback' def test_node_application_write_before_write_head(self): self.load('write_before_write_head') @@ -255,7 +255,7 @@ class TestNodeApplication(TestApplicationNode): )['status'] == 200 ), 'promise end request' - assert waitforfiles(temp_dir + '/node/callback'), 'promise end' + assert waitforfiles(f'{temp_dir}/node/callback'), 'promise end' @pytest.mark.skip('not yet') def test_node_application_header_name_valid(self): diff --git a/test/test_node_websockets.py b/test/test_node_websockets.py index 1f9a2e6b..f1767cac 100644 --- a/test/test_node_websockets.py +++ b/test/test_node_websockets.py @@ -109,7 +109,7 @@ class TestNodeWebsockets(TestApplicationNode): frame = self.ws.frame_read(sock) - assert message + ' ' + message == frame['data'].decode( + assert f'{message} {message}' == frame['data'].decode( 'utf-8' ), 'mirror framing' @@ -473,21 +473,21 @@ class TestNodeWebsockets(TestApplicationNode): _, sock, _ = self.ws.upgrade() for i in range(0, 10): - self.ws.frame_write(sock, self.ws.OP_PING, 'payload-%d' % i) + self.ws.frame_write(sock, self.ws.OP_PING, f'payload-{i}') for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') # 2_11 for i in range(0, 10): opcode = self.ws.OP_PING - self.ws.frame_write(sock, opcode, 'payload-%d' % i, chopsize=1) + self.ws.frame_write(sock, opcode, f'payload-{i}', chopsize=1) for i in range(0, 10): frame = self.ws.frame_read(sock) - self.check_frame(frame, True, self.ws.OP_PONG, 'payload-%d' % i) + self.check_frame(frame, True, self.ws.OP_PONG, f'payload-{i}') self.close_connection(sock) diff --git a/test/test_perl_application.py b/test/test_perl_application.py index fe2db72e..3c327aa1 100644 --- a/test/test_perl_application.py +++ b/test/test_perl_application.py @@ -179,6 +179,11 @@ class TestPerlApplication(TestApplicationPerl): assert self.get()['body'] == 'body\n', 'body io file' + def test_perl_streaming_body_multiple_responses(self): + self.load('streaming_body_multiple_responses') + + assert self.get()['status'] == 200 + @pytest.mark.skip('not yet') def test_perl_application_syntax_error(self, skip_alert): skip_alert(r'PSGI: Failed to parse script') diff --git a/test/test_php_application.py b/test/test_php_application.py index f442f551..6e1d190a 100644 --- a/test/test_php_application.py +++ b/test/test_php_application.py @@ -33,28 +33,23 @@ class TestPHPApplication(TestApplicationPHP): def set_opcache(self, app, val): assert 'success' in self.conf( {"admin": {"opcache.enable": val, "opcache.enable_cli": val}}, - 'applications/' + app + '/options', + f'applications/{app}/options', ) r = self.check_opcache() assert r['headers']['X-OPcache'] == val, 'opcache value' def set_preload(self, preload): - with open(option.temp_dir + '/php.ini', 'w') as f: + with open(f'{option.temp_dir}/php.ini', 'w') as f: f.write( - """opcache.preload = %(test_dir)s/php/opcache/preload\ -/%(preload)s -opcache.preload_user = %(user)s + f"""opcache.preload = {option.test_dir}/php/opcache/preload\ +/{preload} +opcache.preload_user = {option.user or getpass.getuser()} """ - % { - 'test_dir': option.test_dir, - 'preload': preload, - 'user': option.user or getpass.getuser(), - } ) assert 'success' in self.conf( - {"file": option.temp_dir + "/php.ini"}, + {"file": f"{option.temp_dir}/php.ini"}, 'applications/opcache/options', ) @@ -295,7 +290,7 @@ opcache.preload_user = %(user)s assert ( self.get()['headers']['X-File'] - == option.test_dir + '/php/ini_precision/ini/php.ini' + == f'{option.test_dir}/php/ini_precision/ini/php.ini' ), 'ini file' assert self.get()['headers']['X-Precision'] == '4', 'ini value' @@ -318,7 +313,7 @@ opcache.preload_user = %(user)s assert ( self.get()['headers']['X-File'] - == option.test_dir + '/php/ini_precision/ini/php.ini' + == f'{option.test_dir}/php/ini_precision/ini/php.ini' ), 'ini file' assert self.get()['headers']['X-Precision'] == '5', 'ini value admin' @@ -332,7 +327,7 @@ opcache.preload_user = %(user)s assert ( self.get()['headers']['X-File'] - == option.test_dir + '/php/ini_precision/ini/php.ini' + == f'{option.test_dir}/php/ini_precision/ini/php.ini' ), 'ini file' assert self.get()['headers']['X-Precision'] == '5', 'ini value user' @@ -590,7 +585,7 @@ opcache.preload_user = %(user)s "script": { "type": self.get_application_type(), "processes": {"spare": 0}, - "root": option.test_dir + "/php/script", + "root": f"{option.test_dir}/php/script", "script": "phpinfo.php", } }, @@ -610,7 +605,7 @@ opcache.preload_user = %(user)s "phpinfo": { "type": self.get_application_type(), "processes": {"spare": 0}, - "root": option.test_dir + "/php/phpinfo", + "root": f"{option.test_dir}/php/phpinfo", } }, } @@ -622,18 +617,18 @@ opcache.preload_user = %(user)s assert resp['body'] != '', 'body not empty' def test_php_application_trailing_slash(self, temp_dir): - new_root = temp_dir + "/php-root" - os.makedirs(new_root + '/path') + new_root = f'{temp_dir}/php-root' + os.makedirs(f'{new_root}/path') - Path(new_root + '/path/index.php').write_text('<?php echo "OK\n"; ?>') + Path(f'{new_root}/path/index.php').write_text('<?php echo "OK\n"; ?>') - addr = temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( { "listeners": { "*:7080": {"pass": "applications/php-path"}, - "unix:" + addr: {"pass": "applications/php-path"}, + f'unix:{addr}': {"pass": "applications/php-path"}, }, "applications": { "php-path": { @@ -664,14 +659,34 @@ opcache.preload_user = %(user)s resp['headers']['Location'] == 'http://foo/path/' ), 'Location with custom Host over UDS' + def test_php_application_forbidden(self, temp_dir): + new_root = f'{temp_dir}/php-root/path' + os.makedirs(new_root) + os.chmod(new_root, 0o000) + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "applications/php-path"}}, + "applications": { + "php-path": { + "type": self.get_application_type(), + "processes": {"spare": 0}, + "root": f'{temp_dir}/php-root', + } + }, + } + ), 'forbidden directory' + + assert self.get(url='/path/')['status'] == 403, 'access forbidden' + def test_php_application_extension_check(self, temp_dir): self.load('phpinfo') assert self.get(url='/index.wrong')['status'] != 200, 'status' - new_root = temp_dir + "/php" + new_root = f'{temp_dir}/php' os.mkdir(new_root) - shutil.copy(option.test_dir + '/php/phpinfo/index.wrong', new_root) + shutil.copy(f'{option.test_dir}/php/phpinfo/index.wrong', new_root) assert 'success' in self.conf( { @@ -688,21 +703,21 @@ opcache.preload_user = %(user)s ), 'configure new root' resp = self.get() - assert str(resp['status']) + resp['body'] != '200', 'status new root' + assert f'{resp["status"]}{resp["body"]}' != '200', 'status new root' def run_php_application_cwd_root_tests(self): assert 'success' in self.conf_delete( 'applications/cwd/working_directory' ) - script_cwd = option.test_dir + '/php/cwd' + script_cwd = f'{option.test_dir}/php/cwd' resp = self.get() assert resp['status'] == 200, 'status ok' assert resp['body'] == script_cwd, 'default cwd' assert 'success' in self.conf( - '"' + option.test_dir + '"', + f'"{option.test_dir}"', 'applications/cwd/working_directory', ) @@ -721,7 +736,7 @@ opcache.preload_user = %(user)s assert resp['body'] == script_cwd, 'cwd restored' resp = self.get(url='/subdir/') - assert resp['body'] == script_cwd + '/subdir', 'cwd subdir' + assert resp['body'] == f'{script_cwd}/subdir', 'cwd subdir' def test_php_application_cwd_root(self): self.load('cwd') @@ -740,7 +755,7 @@ opcache.preload_user = %(user)s def run_php_application_cwd_script_tests(self): self.load('cwd') - script_cwd = option.test_dir + '/php/cwd' + script_cwd = f'{option.test_dir}/php/cwd' assert 'success' in self.conf_delete( 'applications/cwd/working_directory' diff --git a/test/test_php_targets.py b/test/test_php_targets.py index eec1846f..e74f2ec6 100644 --- a/test/test_php_targets.py +++ b/test/test_php_targets.py @@ -6,6 +6,7 @@ class TestPHPTargets(TestApplicationPHP): prerequisites = {'modules': {'php': 'any'}} def test_php_application_targets(self): + targets_dir = f"{option.test_dir}/php/targets" assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, @@ -27,15 +28,15 @@ class TestPHPTargets(TestApplicationPHP): "targets": { "1": { "script": "1.php", - "root": option.test_dir + "/php/targets", + "root": targets_dir, }, "2": { "script": "2.php", - "root": option.test_dir + "/php/targets/2", + "root": f'{targets_dir}/2', }, "default": { "index": "index.php", - "root": option.test_dir + "/php/targets", + "root": targets_dir, }, }, } @@ -45,7 +46,7 @@ class TestPHPTargets(TestApplicationPHP): assert self.get(url='/1')['body'] == '1' assert self.get(url='/2')['body'] == '2' - assert self.get(url='/blah')['status'] == 503 # TODO 404 + assert self.get(url='/blah')['status'] == 404 assert self.get(url='/')['body'] == 'index' assert self.get(url='/1.php?test=test.php/')['body'] == '1' @@ -72,7 +73,7 @@ class TestPHPTargets(TestApplicationPHP): "targets": { "default": { "index": "index.php", - "root": option.test_dir + "/php/targets", + "root": f"{option.test_dir}/php/targets", }, }, } @@ -85,7 +86,7 @@ class TestPHPTargets(TestApplicationPHP): {"pass": "applications/targets/blah"}, 'listeners/*:7080' ), 'invalid targets pass' assert 'error' in self.conf( - '"' + option.test_dir + '/php/targets\"', + f'"{option.test_dir}/php/targets"', 'applications/targets/root', ), 'invalid root' assert 'error' in self.conf( diff --git a/test/test_proxy.py b/test/test_proxy.py index ede91fd6..74e48ca1 100644 --- a/test/test_proxy.py +++ b/test/test_proxy.py @@ -63,6 +63,7 @@ Content-Length: 10 run_process(self.run_server, self.SERVER_PORT) waitforsocket(self.SERVER_PORT) + python_dir = f'{option.test_dir}/python' assert 'success' in self.conf( { "listeners": { @@ -74,24 +75,22 @@ Content-Length: 10 "mirror": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/mirror", - "working_directory": option.test_dir + "/python/mirror", + "path": f'{python_dir}/mirror', + "working_directory": f'{python_dir}/mirror', "module": "wsgi", }, "custom_header": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/custom_header", - "working_directory": option.test_dir - + "/python/custom_header", + "path": f'{python_dir}/custom_header', + "working_directory": f'{python_dir}/custom_header', "module": "wsgi", }, "delayed": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/delayed", - "working_directory": option.test_dir - + "/python/delayed", + "path": f'{python_dir}/delayed', + "working_directory": f'{python_dir}/delayed', "module": "wsgi", }, }, @@ -124,8 +123,8 @@ Content-Length: 10 "mirror": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/mirror", - "working_directory": option.test_dir + "/python/mirror", + "path": f'{option.test_dir}/python/mirror', + "working_directory": f'{option.test_dir}/python/mirror', "module": "wsgi", } }, @@ -186,7 +185,7 @@ Content-Length: 10 socks = [] for i in range(10): sock = self.post_http10( - body=payload + str(i), + body=f'{payload}{i}', no_recv=True, read_buffer_size=buff_size, ) @@ -199,7 +198,7 @@ Content-Length: 10 resp = self._resp_to_dict(resp) assert resp['status'] == 200, 'status' - assert resp['body'] == payload + str(i), 'body' + assert resp['body'] == f'{payload}{i}', 'body' def test_proxy_header(self): assert 'success' in self.conf( @@ -214,7 +213,7 @@ Content-Length: 10 == header_value ), 'custom header' - header_value = r'(),/:;<=>?@[\]{}\t !#$%&\'*+-.^_`|~' + header_value = r"(),/:;<=>?@[\]{}\t !#$%&'*+-.^_`|~" assert ( self.get_http10( headers={'Host': 'localhost', 'Custom-Header': header_value} @@ -336,18 +335,18 @@ Content-Length: 10 assert self.get_http10()['status'] == 200, 'status' def test_proxy_unix(self, temp_dir): - addr = temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( { "*:7080": {"pass": "routes"}, - "unix:" + addr: {'application': 'mirror'}, + f'unix:{addr}': {'application': 'mirror'}, }, 'listeners', ), 'add unix listener configure' assert 'success' in self.conf( - [{"action": {"proxy": 'http://unix:' + addr}}], 'routes' + [{"action": {"proxy": f'http://unix:{addr}'}}], 'routes' ), 'proxy unix configure' assert self.get_http10()['status'] == 200, 'status' @@ -420,13 +419,7 @@ Content-Length: 10 @pytest.mark.skip('not yet') def test_proxy_content_length(self): assert 'success' in self.conf( - [ - { - "action": { - "proxy": "http://127.0.0.1:" + str(self.SERVER_PORT) - } - } - ], + [{"action": {"proxy": f'http://127.0.0.1:{self.SERVER_PORT}'}}], 'routes', ), 'proxy backend configure' @@ -484,8 +477,8 @@ Content-Length: 10 "mirror": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/mirror", - "working_directory": option.test_dir + "/python/mirror", + "path": f'{option.test_dir}/python/mirror', + "working_directory": f'{option.test_dir}/python/mirror', "module": "wsgi", }, }, diff --git a/test/test_proxy_chunked.py b/test/test_proxy_chunked.py index f024eaf5..f31c976a 100644 --- a/test/test_proxy_chunked.py +++ b/test/test_proxy_chunked.py @@ -61,7 +61,7 @@ class TestProxyChunked(TestApplicationPython): else: add = line - req = req + add + '\r\n' + req = f'{req}{add}\r\n' for chunk in re.split(r'([@#])', req): if chunk == '@' or chunk == '#': @@ -77,9 +77,9 @@ class TestProxyChunked(TestApplicationPython): body = '\r\n\r\n' for l, c in chunks: - body = body + l + '\r\n' + c + '\r\n' + body = f'{body}{l}\r\n{c}\r\n' - return body + '0\r\n\r\n' + return f'{body}0\r\n\r\n' def get_http10(self, *args, **kwargs): return self.get(*args, http_10=True, **kwargs) @@ -96,7 +96,7 @@ class TestProxyChunked(TestApplicationPython): "routes": [ { "action": { - "proxy": "http://127.0.0.1:" + str(self.SERVER_PORT) + "proxy": f'http://127.0.0.1:{self.SERVER_PORT}' } } ], @@ -111,20 +111,20 @@ class TestProxyChunked(TestApplicationPython): part = '0123456789abcdef' assert ( - self.get_http10(body=self.chunks([('1000', part + ' X 256')]))[ + self.get_http10(body=self.chunks([('1000', f'{part} X 256')]))[ 'body' ] == part * 256 ) assert ( - self.get_http10(body=self.chunks([('100000', part + ' X 65536')]))[ + self.get_http10(body=self.chunks([('100000', f'{part} X 65536')]))[ 'body' ] == part * 65536 ) assert ( self.get_http10( - body=self.chunks([('1000000', part + ' X 1048576')]), + body=self.chunks([('1000000', f'{part} X 1048576')]), read_buffer_size=4096 * 4096, )['body'] == part * 1048576 @@ -133,7 +133,7 @@ class TestProxyChunked(TestApplicationPython): assert ( self.get_http10( body=self.chunks( - [('1000', part + ' X 256'), ('1000', part + ' X 256')] + [('1000', f'{part} X 256'), ('1000', f'{part} X 256')] ) )['body'] == part * 256 * 2 @@ -142,8 +142,8 @@ class TestProxyChunked(TestApplicationPython): self.get_http10( body=self.chunks( [ - ('100000', part + ' X 65536'), - ('100000', part + ' X 65536'), + ('100000', f'{part} X 65536'), + ('100000', f'{part} X 65536'), ] ) )['body'] @@ -153,8 +153,8 @@ class TestProxyChunked(TestApplicationPython): self.get_http10( body=self.chunks( [ - ('1000000', part + ' X 1048576'), - ('1000000', part + ' X 1048576'), + ('1000000', f'{part} X 1048576'), + ('1000000', f'{part} X 1048576'), ] ), read_buffer_size=4096 * 4096, diff --git a/test/test_python_application.py b/test/test_python_application.py index c9065eae..d412ac68 100644 --- a/test/test_python_application.py +++ b/test/test_python_application.py @@ -20,17 +20,16 @@ class TestPythonApplication(TestApplicationPython): body = 'Test body string.' resp = self.http( - b"""POST / HTTP/1.1 + f"""POST / HTTP/1.1 Host: localhost -Content-Length: %d +Content-Length: {len(body)} Custom-Header: blah Custom-hEader: Blah Content-Type: text/html Connection: close custom-header: BLAH -%s""" - % (len(body), body.encode()), +{body}""".encode(), raw=True, ) @@ -101,7 +100,7 @@ custom-header: BLAH self.load('prefix', prefix='/api/rest') def set_prefix(prefix): - self.conf('"' + prefix + '"', 'applications/prefix/prefix') + self.conf(f'"{prefix}"', 'applications/prefix/prefix') def check_prefix(url, script_name, path_info): resp = self.get(url=url) @@ -585,12 +584,12 @@ last line: 987654321 except version.InvalidVersion: pytest.skip('require python module version 3') - venv_path = temp_dir + '/venv' + venv_path = f'{temp_dir}/venv' venv.create(venv_path) self.load('unicode') assert 'success' in self.conf( - '"' + venv_path + '"', + f'"{venv_path}"', '/config/applications/unicode/home', ) assert ( @@ -640,7 +639,7 @@ last line: 987654321 assert self.wait_for_record(r'Traceback') is not None, 'traceback' assert ( - self.wait_for_record(r'raise Exception\(\'first exception\'\)') + self.wait_for_record(r"raise Exception\('first exception'\)") is not None ), 'first exception raise' assert len(self.findall(r'Traceback')) == 1, 'traceback count 1' @@ -659,7 +658,7 @@ last line: 987654321 ), 'error 2' assert ( - self.wait_for_record(r'raise Exception\(\'second exception\'\)') + self.wait_for_record(r"raise Exception\('second exception'\)") is not None ), 'exception raise second' assert len(self.findall(r'Traceback')) == 2, 'traceback count 2' @@ -676,7 +675,7 @@ last line: 987654321 ) assert ( - self.wait_for_record(r'raise Exception\(\'third exception\'\)') + self.wait_for_record(r"raise Exception\('third exception'\)") is not None ), 'exception raise third' assert len(self.findall(r'Traceback')) == 3, 'traceback count 3' @@ -711,7 +710,7 @@ last line: 987654321 ) assert ( - self.wait_for_record(r'raise Exception\(\'next exception\'\)') + self.wait_for_record(r"raise Exception\('next exception'\)") is not None ), 'exception raise next' assert len(self.findall(r'Traceback')) == 5, 'traceback count 5' @@ -747,7 +746,7 @@ last line: 987654321 ), 'error' assert ( - self.wait_for_record(r'raise Exception\(\'close exception\'\)') + self.wait_for_record(r"raise Exception\('close exception'\)") is not None ), 'exception raise close' assert len(self.findall(r'Traceback')) == 8, 'traceback count 8' @@ -781,18 +780,14 @@ last line: 987654321 self.load('user_group', user='nobody', group=group) obj = self.getjson()['body'] - assert obj['UID'] == nobody_uid, ( - 'nobody uid user=nobody group=%s' % group - ) - - assert obj['GID'] == group_id, 'nobody gid user=nobody group=%s' % group + assert obj['UID'] == nobody_uid, f'nobody uid user=nobody group={group}' + assert obj['GID'] == group_id, f'nobody gid user=nobody group={group}' self.load('user_group', group=group) obj = self.getjson()['body'] - assert obj['UID'] == nobody_uid, 'nobody uid group=%s' % group - - assert obj['GID'] == group_id, 'nobody gid group=%s' % group + assert obj['UID'] == nobody_uid, f'nobody uid group={group}' + assert obj['GID'] == group_id, f'nobody gid group={group}' self.load('user_group', user='root') diff --git a/test/test_python_environment.py b/test/test_python_environment.py index 2d7d1595..bce72c4d 100644 --- a/test/test_python_environment.py +++ b/test/test_python_environment.py @@ -38,7 +38,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == 'val1,' + == 'val1' ), 'set' self.conf({"var": "val2"}, 'applications/environment/environment') @@ -51,7 +51,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == 'val2,' + == 'val2' ), 'update' def test_python_environment_replace(self): @@ -67,7 +67,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == 'val1,' + == 'val1' ), 'set' self.conf({"var2": "val2"}, 'applications/environment/environment') @@ -80,7 +80,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == 'val2,' + == 'val2' ), 'replace' def test_python_environment_clear(self): @@ -99,7 +99,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == 'val1,val2,' + == 'val1,val2' ), 'set' self.conf({}, 'applications/environment/environment') @@ -138,7 +138,7 @@ class TestPythonEnvironment(TestApplicationPython): 'Connection': 'close', } )['body'] - == '/,' + == '/' ), 'replace default' self.conf({}, 'applications/environment/environment') diff --git a/test/test_python_isolation.py b/test/test_python_isolation.py index 6d4ffaf3..c524aea0 100644 --- a/test/test_python_isolation.py +++ b/test/test_python_isolation.py @@ -20,13 +20,13 @@ class TestPythonIsolation(TestApplicationPython): ).decode() pid = re.search( - r'(\d+)\s*unit: "' + app_name + '" application', output + fr'(\d+)\s*unit: "{app_name}" application', output ).group(1) - cgroup = '/proc/' + pid + '/cgroup' + cgroup = f'/proc/{pid}/cgroup' if not os.path.isfile(cgroup): - pytest.skip('no cgroup at ' + cgroup) + pytest.skip(f'no cgroup at {cgroup}') with open(cgroup, 'r') as f: return f.read().rstrip() @@ -59,7 +59,7 @@ class TestPythonIsolation(TestApplicationPython): self.load('ns_inspect', isolation=isolation) assert ( - self.getjson(url='/?path=' + temp_dir)['body']['FileExists'] + self.getjson(url=f'/?path={temp_dir}')['body']['FileExists'] == False ), 'temp_dir does not exists in rootfs' @@ -87,7 +87,7 @@ class TestPythonIsolation(TestApplicationPython): isolation = {'rootfs': temp_dir, 'automount': {'language_deps': False}} self.load('empty', isolation=isolation) - python_path = temp_dir + '/usr' + python_path = f'{temp_dir}/usr' assert findmnt().find(python_path) == -1 assert self.get()['status'] != 200, 'disabled language_deps' @@ -156,7 +156,7 @@ class TestPythonIsolation(TestApplicationPython): pytest.skip('cgroup is not supported') def set_two_cgroup_path(path, path2): - script_path = option.test_dir + '/python/empty' + script_path = f'{option.test_dir}/python/empty' assert 'success' in self.conf( { @@ -203,7 +203,7 @@ class TestPythonIsolation(TestApplicationPython): pytest.skip('cgroup is not supported') def check_invalid(path): - script_path = option.test_dir + '/python/empty' + script_path = f'{option.test_dir}/python/empty' assert 'error' in self.conf( { "listeners": {"*:7080": {"pass": "applications/empty"}}, diff --git a/test/test_python_isolation_chroot.py b/test/test_python_isolation_chroot.py index 8e5b5fce..349ec869 100644 --- a/test/test_python_isolation_chroot.py +++ b/test/test_python_isolation_chroot.py @@ -16,7 +16,7 @@ class TestPythonIsolation(TestApplicationPython): self.load('ns_inspect', isolation=isolation) assert ( - self.getjson(url='/?path=' + temp_dir)['body']['FileExists'] + self.getjson(url=f'/?path={temp_dir}')['body']['FileExists'] == False ), 'temp_dir does not exists in rootfs' diff --git a/test/test_python_procman.py b/test/test_python_procman.py index a25b84ec..d69123ef 100644 --- a/test/test_python_procman.py +++ b/test/test_python_procman.py @@ -12,8 +12,8 @@ class TestPythonProcman(TestApplicationPython): prerequisites = {'modules': {'python': 'any'}} def setup_method(self): - self.app_name = "app-" + option.temp_dir.split('/')[-1] - self.app_proc = 'applications/' + self.app_name + '/processes' + self.app_name = f'app-{option.temp_dir.split("/")[-1]}' + self.app_proc = f'applications/{self.app_name}/processes' self.load('empty', self.app_name) def pids_for_process(self): @@ -23,7 +23,7 @@ class TestPythonProcman(TestApplicationPython): pids = set() for m in re.findall( - '.*unit: "' + self.app_name + '" application', output.decode() + fr'.*unit: "{self.app_name}" application', output.decode() ): pids.add(re.search(r'^\s*(\d+)', m).group(1)) @@ -126,7 +126,7 @@ class TestPythonProcman(TestApplicationPython): assert len(pids_new) == 3, 'reconf 3' assert pids.issubset(pids_new), 'reconf 3 only 1 new' - self.conf_proc('6', self.app_proc + '/spare') + self.conf_proc('6', f'{self.app_proc}/spare') pids = self.pids_for_process() assert len(pids) == 6, 'reconf 6' @@ -176,10 +176,10 @@ class TestPythonProcman(TestApplicationPython): def test_python_processes_access(self): self.conf_proc('1') - path = '/' + self.app_proc - assert 'error' in self.conf_get(path + '/max') - assert 'error' in self.conf_get(path + '/spare') - assert 'error' in self.conf_get(path + '/idle_timeout') + path = f'/{self.app_proc}' + assert 'error' in self.conf_get(f'{path}/max') + assert 'error' in self.conf_get(f'{path}/spare') + assert 'error' in self.conf_get(f'{path}/idle_timeout') def test_python_processes_invalid(self): assert 'error' in self.conf( @@ -206,7 +206,7 @@ class TestPythonProcman(TestApplicationPython): def test_python_restart(self, temp_dir): shutil.copyfile( - option.test_dir + '/python/restart/v1.py', temp_dir + '/wsgi.py' + f'{option.test_dir}/python/restart/v1.py', f'{temp_dir}/wsgi.py' ) self.load( @@ -220,14 +220,14 @@ class TestPythonProcman(TestApplicationPython): assert b == "v1", 'process started' shutil.copyfile( - option.test_dir + '/python/restart/v2.py', temp_dir + '/wsgi.py' + f'{option.test_dir}/python/restart/v2.py', f'{temp_dir}/wsgi.py' ) b = self.get()['body'] assert b == "v1", 'still old process' assert 'success' in self.conf_get( - '/control/applications/' + self.app_name + '/restart' + f'/control/applications/{self.app_name}/restart' ), 'restart processes' b = self.get()['body'] @@ -238,7 +238,7 @@ class TestPythonProcman(TestApplicationPython): ), 'application incorrect' assert 'error' in self.conf_delete( - '/control/applications/' + self.app_name + '/restart' + f'/control/applications/{self.app_name}/restart' ), 'method incorrect' def test_python_restart_multi(self): @@ -248,7 +248,7 @@ class TestPythonProcman(TestApplicationPython): assert len(pids) == 2, 'restart 2 started' assert 'success' in self.conf_get( - '/control/applications/' + self.app_name + '/restart' + f'/control/applications/{self.app_name}/restart' ), 'restart processes' new_pids = self.pids_for_process() @@ -272,7 +272,7 @@ class TestPythonProcman(TestApplicationPython): assert len(pids) == 2, 'longstarts == 2' assert 'success' in self.conf_get( - '/control/applications/' + self.app_name + '/restart' + f'/control/applications/{self.app_name}/restart' ), 'restart processes' # wait for longstarted app diff --git a/test/test_python_targets.py b/test/test_python_targets.py index ae271b5f..f55609ba 100644 --- a/test/test_python_targets.py +++ b/test/test_python_targets.py @@ -6,6 +6,8 @@ class TestPythonTargets(TestApplicationPython): prerequisites = {'modules': {'python': 'all'}} def test_python_targets(self): + python_dir = f'{option.test_dir}/python' + assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, @@ -22,9 +24,8 @@ class TestPythonTargets(TestApplicationPython): "applications": { "targets": { "type": self.get_application_type(), - "working_directory": option.test_dir - + "/python/targets/", - "path": option.test_dir + '/python/targets/', + "working_directory": f'{python_dir}/targets/', + "path": f'{python_dir}/targets/', "targets": { "1": { "module": "wsgi", @@ -49,6 +50,8 @@ class TestPythonTargets(TestApplicationPython): assert resp['body'] == '2' def test_python_targets_prefix(self): + python_dir = f'{option.test_dir}/python' + assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, @@ -65,9 +68,8 @@ class TestPythonTargets(TestApplicationPython): "applications": { "targets": { "type": "python", - "working_directory": option.test_dir - + "/python/targets/", - "path": option.test_dir + '/python/targets/', + "working_directory": f'{python_dir}/targets/', + "path": f'{python_dir}/targets/', "protocol": "wsgi", "targets": { "app": { diff --git a/test/test_respawn.py b/test/test_respawn.py index 19d97d37..3d3dfac3 100644 --- a/test/test_respawn.py +++ b/test/test_respawn.py @@ -13,21 +13,21 @@ class TestRespawn(TestApplicationPython): PATTERN_CONTROLLER = 'unit: controller' def setup_method(self): - self.app_name = "app-" + option.temp_dir.split('/')[-1] + self.app_name = f'app-{option.temp_dir.split("/")[-1]}' self.load('empty', self.app_name) assert 'success' in self.conf( - '1', 'applications/' + self.app_name + '/processes' + '1', f'applications/{self.app_name}/processes' ) def pid_by_name(self, name, ppid): output = subprocess.check_output(['ps', 'ax', '-O', 'ppid']).decode() - m = re.search(r'\s*(\d+)\s*' + str(ppid) + r'.*' + name, output) + m = re.search(fr'\s*(\d+)\s*{ppid}.*{name}', output) return None if m is None else m.group(1) def kill_pids(self, *pids): - subprocess.call(['kill', '-9'] + list(pids)) + subprocess.call(['kill', '-9', *pids]) def wait_for_process(self, process, unit_pid): for i in range(50): @@ -41,11 +41,11 @@ class TestRespawn(TestApplicationPython): return found def find_proc(self, name, ppid, ps_output): - return re.findall(str(ppid) + r'.*' + name, ps_output) + return re.findall(fr'{ppid}.*{name}', ps_output) def smoke_test(self, unit_pid): for _ in range(10): - r = self.conf('1', 'applications/' + self.app_name + '/processes') + r = self.conf('1', f'applications/{self.app_name}/processes') if 'success' in r: break @@ -68,7 +68,7 @@ class TestRespawn(TestApplicationPython): pid = self.pid_by_name(self.PATTERN_ROUTER, unit_pid) self.kill_pids(pid) - skip_alert(r'process %s exited on signal 9' % pid) + skip_alert(fr'process {pid} exited on signal 9') assert self.wait_for_process(self.PATTERN_ROUTER, unit_pid) is not None @@ -79,7 +79,7 @@ class TestRespawn(TestApplicationPython): pid = self.pid_by_name(self.PATTERN_CONTROLLER, unit_pid) self.kill_pids(pid) - skip_alert(r'process %s exited on signal 9' % pid) + skip_alert(fr'process {pid} exited on signal 9') assert ( self.wait_for_process(self.PATTERN_CONTROLLER, unit_pid) is not None @@ -93,7 +93,7 @@ class TestRespawn(TestApplicationPython): pid = self.pid_by_name(self.app_name, unit_pid) self.kill_pids(pid) - skip_alert(r'process %s exited on signal 9' % pid) + skip_alert(fr'process {pid} exited on signal 9') assert self.wait_for_process(self.app_name, unit_pid) is not None diff --git a/test/test_return.py b/test/test_return.py index 82bf1e64..4b8bddc7 100644 --- a/test/test_return.py +++ b/test/test_return.py @@ -107,26 +107,26 @@ Connection: close check_location(reserved) # After first "?" all other "?" encoded. - check_location("/?" + reserved, "/?:/%3F#[]@!&'()*+,;=") + check_location(f'/?{reserved}', "/?:/%3F#[]@!&'()*+,;=") check_location("???", "?%3F%3F") # After first "#" all other "?" or "#" encoded. - check_location("/#" + reserved, "/#:/%3F%23[]@!&'()*+,;=") + check_location(f'/#{reserved}', "/#:/%3F%23[]@!&'()*+,;=") check_location("##?#?", "#%23%3F%23%3F") # After first "?" next "#" not encoded. - check_location("/?#" + reserved, "/?#:/%3F%23[]@!&'()*+,;=") + check_location(f'/?#{reserved}', "/?#:/%3F%23[]@!&'()*+,;=") check_location("??##", "?%3F#%23") check_location("/?##?", "/?#%23%3F") # Unreserved never encoded. check_location(unreserved) - check_location("/" + unreserved + "?" + unreserved + "#" + unreserved) + check_location(f'/{unreserved}?{unreserved}#{unreserved}') # Unsafe always encoded. check_location(unsafe, unsafe_enc) - check_location("?" + unsafe, "?" + unsafe_enc) - check_location("#" + unsafe, "#" + unsafe_enc) + check_location(f'?{unsafe}', f'?{unsafe_enc}') + check_location(f'#{unsafe}', f'#{unsafe_enc}') # %00-%20 and %7F-%FF always encoded. check_location(u"\u0000\u0018\u001F\u0020\u0021", "%00%18%1F%20!") diff --git a/test/test_rewrite.py b/test/test_rewrite.py new file mode 100644 index 00000000..3bc7df19 --- /dev/null +++ b/test/test_rewrite.py @@ -0,0 +1,219 @@ +import os + +import pytest +from unit.applications.proto import TestApplicationProto +from unit.option import option + + +class TestRewrite(TestApplicationProto): + prerequisites = {} + + def setup_method(self): + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "match": {"uri": "/"}, + "action": {"rewrite": "/new", "pass": "routes"}, + }, + {"match": {"uri": "/new"}, "action": {"return": 200}}, + ], + "applications": {}, + "settings": {"http": {"log_route": True}}, + }, + ), 'set initial configuration' + + def set_rewrite(self, rewrite, uri): + assert 'success' in self.conf( + [ + { + "match": {"uri": "/"}, + "action": {"rewrite": rewrite, "pass": "routes"}, + }, + {"match": {"uri": uri}, "action": {"return": 200}}, + ], + 'routes', + ) + + def test_rewrite(self): + assert self.get()['status'] == 200 + assert ( + self.wait_for_record(rf'\[notice\].*"routes/1" selected') + is not None + ) + assert len(self.findall(rf'\[notice\].*URI rewritten to "/new"')) == 1 + assert len(self.findall(rf'\[notice\].*URI rewritten')) == 1 + + self.set_rewrite("", "") + assert self.get()['status'] == 200 + + def test_rewrite_variable(self): + self.set_rewrite("/$host", "/localhost") + assert self.get()['status'] == 200 + + self.set_rewrite("${uri}a", "/a") + assert self.get()['status'] == 200 + + def test_rewrite_encoded(self): + assert 'success' in self.conf( + [ + { + "match": {"uri": "/f"}, + "action": {"rewrite": "${request_uri}oo", "pass": "routes"}, + }, + {"match": {"uri": "/foo"}, "action": {"return": 200}}, + ], + 'routes', + ) + assert self.get(url='/%66')['status'] == 200 + + assert 'success' in self.conf( + [ + { + "match": {"uri": "/f"}, + "action": { + "rewrite": "${request_uri}o%6F", + "pass": "routes", + }, + }, + {"match": {"uri": "/foo"}, "action": {"return": 200}}, + ], + 'routes', + ) + assert self.get(url='/%66')['status'] == 200 + + def test_rewrite_arguments(self): + assert 'success' in self.conf( + [ + { + "match": {"uri": "/foo", "arguments": {"arg": "val"}}, + "action": {"rewrite": "/new?some", "pass": "routes"}, + }, + { + "match": {"uri": "/new", "arguments": {"arg": "val"}}, + "action": {"return": 200}, + }, + ], + 'routes', + ) + assert self.get(url='/foo?arg=val')['status'] == 200 + + def test_rewrite_njs(self): + if 'njs' not in option.available['modules'].keys(): + pytest.skip('NJS is not available') + + self.set_rewrite("`/${host}`", "/localhost") + assert self.get()['status'] == 200 + + def test_rewrite_location(self): + def check_location(rewrite, expect): + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "action": { + "return": 301, + "location": "$uri", + "rewrite": rewrite, + } + } + ], + } + ) + assert self.get()['headers']['Location'] == expect + + check_location('/new', '/new') + check_location('${request_uri}new', '/new') + + def test_rewrite_share(self, temp_dir): + os.makedirs(f'{temp_dir}/dir') + os.makedirs(f'{temp_dir}/foo') + + with open(f'{temp_dir}/foo/index.html', 'w') as fooindex: + fooindex.write('fooindex') + + # same action block + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "action": { + "rewrite": "${request_uri}dir", + "share": f'{temp_dir}$uri', + } + } + ], + } + ) + + resp = self.get() + assert resp['status'] == 301, 'redirect status' + assert resp['headers']['Location'] == '/dir/', 'redirect Location' + + # request_uri + + index_path = f'{temp_dir}${{request_uri}}/index.html' + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "match": {"uri": "/foo"}, + "action": { + "rewrite": "${request_uri}dir", + "pass": "routes", + }, + }, + {"action": {"share": index_path}}, + ], + } + ) + + assert self.get(url='/foo')['body'] == 'fooindex' + + # different action block + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "match": {"uri": "/foo"}, + "action": { + "rewrite": "${request_uri}dir", + "pass": "routes", + }, + }, + { + "action": { + "share": f'{temp_dir}/dir', + } + }, + ], + } + ) + resp = self.get(url='/foo') + assert resp['status'] == 301, 'redirect status 2' + assert resp['headers']['Location'] == '/foodir/', 'redirect Location 2' + + def test_rewrite_invalid(self, skip_alert): + skip_alert(r'failed to apply new conf') + + def check_rewrite(rewrite): + assert 'error' in self.conf( + [ + { + "match": {"uri": "/"}, + "action": {"rewrite": rewrite, "pass": "routes"}, + }, + {"action": {"return": 200}}, + ], + 'routes', + ) + + check_rewrite("/$blah") + check_rewrite(["/"]) diff --git a/test/test_routing.py b/test/test_routing.py index 9e872061..a4806d5c 100644 --- a/test/test_routing.py +++ b/test/test_routing.py @@ -286,17 +286,18 @@ class TestRouting(TestApplicationPython): assert self.get(url='/BLAH')['status'] == 200, '/BLAH' def test_routes_pass_encode(self): + python_dir = f'{option.test_dir}/python' + def check_pass(path, name): assert 'success' in self.conf( { - "listeners": {"*:7080": {"pass": "applications/" + path}}, + "listeners": {"*:7080": {"pass": f'applications/{path}'}}, "applications": { name: { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + '/python/empty', - "working_directory": option.test_dir - + '/python/empty', + "path": f'{python_dir}/empty', + "working_directory": f'{python_dir}/empty', "module": "wsgi", } }, @@ -313,14 +314,13 @@ class TestRouting(TestApplicationPython): def check_pass_error(path, name): assert 'error' in self.conf( { - "listeners": {"*:7080": {"pass": "applications/" + path}}, + "listeners": {"*:7080": {"pass": f'applications/{path}'}}, "applications": { name: { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + '/python/empty', - "working_directory": option.test_dir - + '/python/empty', + "path": f'{python_dir}/empty', + "working_directory": f'{python_dir}/empty', "module": "wsgi", } }, @@ -338,8 +338,8 @@ class TestRouting(TestApplicationPython): "empty": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + '/python/empty', - "working_directory": option.test_dir + '/python/empty', + "path": f'{option.test_dir}/python/empty', + "working_directory": f'{option.test_dir}/python/empty', "module": "wsgi", } }, @@ -1109,21 +1109,21 @@ class TestRouting(TestApplicationPython): "E", "F", ]: - chars_enc += "%" + h1 + h2 + chars_enc += f'%{h1}{h2}' chars_enc = chars_enc[:-3] def check_args(args, query): self.route_match({"arguments": args}) - assert self.get(url='/?' + query)['status'] == 200 + assert self.get(url=f'/?{query}')['status'] == 200 - check_args({chars: chars}, chars + '=' + chars) - check_args({chars: chars}, chars + '=' + chars_enc) - check_args({chars: chars}, chars_enc + '=' + chars) - check_args({chars: chars}, chars_enc + '=' + chars_enc) - check_args({chars_enc: chars_enc}, chars + '=' + chars) - check_args({chars_enc: chars_enc}, chars + '=' + chars_enc) - check_args({chars_enc: chars_enc}, chars_enc + '=' + chars) - check_args({chars_enc: chars_enc}, chars_enc + '=' + chars_enc) + check_args({chars: chars}, f'{chars}={chars}') + check_args({chars: chars}, f'{chars}={chars_enc}') + check_args({chars: chars}, f'{chars_enc}={chars}') + check_args({chars: chars}, f'{chars_enc}={chars_enc}') + check_args({chars_enc: chars_enc}, f'{chars}={chars}') + check_args({chars_enc: chars_enc}, f'{chars}={chars_enc}') + check_args({chars_enc: chars_enc}, f'{chars_enc}={chars}') + check_args({chars_enc: chars_enc}, f'{chars_enc}={chars_enc}') def test_routes_match_arguments_empty(self): self.route_match({"arguments": {}}) @@ -1492,28 +1492,28 @@ class TestRouting(TestApplicationPython): sock, port = sock_port() sock2, port2 = sock_port() - self.route_match({"source": "127.0.0.1:" + str(port)}) + self.route_match({"source": f'127.0.0.1:{port}'}) assert self.get(sock=sock)['status'] == 200, 'exact' assert self.get(sock=sock2)['status'] == 404, 'exact 2' sock, port = sock_port() sock2, port2 = sock_port() - self.route_match({"source": "!127.0.0.1:" + str(port)}) + self.route_match({"source": f'!127.0.0.1:{port}'}) assert self.get(sock=sock)['status'] == 404, 'negative' assert self.get(sock=sock2)['status'] == 200, 'negative 2' sock, port = sock_port() sock2, port2 = sock_port() - self.route_match({"source": ["*:" + str(port), "!127.0.0.1"]}) + self.route_match({"source": [f'*:{port}', "!127.0.0.1"]}) assert self.get(sock=sock)['status'] == 404, 'negative 3' assert self.get(sock=sock2)['status'] == 404, 'negative 4' sock, port = sock_port() sock2, port2 = sock_port() - self.route_match({"source": "127.0.0.1:" + str(port) + "-" + str(port)}) + self.route_match({"source": f'127.0.0.1:{port}-{port}'}) assert self.get(sock=sock)['status'] == 200, 'range single' assert self.get(sock=sock2)['status'] == 404, 'range single 2' @@ -1526,14 +1526,7 @@ class TestRouting(TestApplicationPython): ] socks.sort(key=lambda sock: sock[1]) - self.route_match( - { - "source": "127.0.0.1:" - + str(socks[1][1]) # second port number - + "-" - + str(socks[3][1]) # fourth port number - } - ) + self.route_match({"source": f'127.0.0.1:{socks[1][1]}-{socks[3][1]}'}) assert self.get(sock=socks[0][0])['status'] == 404, 'range' assert self.get(sock=socks[1][0])['status'] == 200, 'range 2' assert self.get(sock=socks[2][0])['status'] == 200, 'range 3' @@ -1550,8 +1543,8 @@ class TestRouting(TestApplicationPython): self.route_match( { "source": [ - "127.0.0.1:" + str(socks[0][1]), - "127.0.0.1:" + str(socks[2][1]), + f'127.0.0.1:{socks[0][1]}', + f'127.0.0.1:{socks[2][1]}', ] } ) @@ -1734,12 +1727,12 @@ class TestRouting(TestApplicationPython): assert self.get(port=7081)['status'] == 404, '0 ipv4' def test_routes_source_unix(self, temp_dir): - addr = temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( { "127.0.0.1:7081": {"pass": "routes"}, - "unix:" + addr: {"pass": "routes"}, + f'unix:{addr}': {"pass": "routes"}, }, 'listeners', ), 'source listeners configure' diff --git a/test/test_ruby_hooks.py b/test/test_ruby_hooks.py index b4a79ebb..078e5723 100644 --- a/test/test_ruby_hooks.py +++ b/test/test_ruby_hooks.py @@ -8,7 +8,7 @@ class TestRubyHooks(TestApplicationRuby): def _wait_cookie(self, pattern, count): return waitforglob( - option.temp_dir + '/ruby/hooks/cookie_' + pattern, count + f'{option.temp_dir}/ruby/hooks/cookie_{pattern}', count ) def test_ruby_hooks_eval(self): diff --git a/test/test_settings.py b/test/test_settings.py index ad8929f8..21ab22d9 100644 --- a/test/test_settings.py +++ b/test/test_settings.py @@ -44,7 +44,7 @@ class TestSettings(TestApplicationPython): headers = {'Host': 'localhost', 'Connection': 'close'} for i in range(headers_num): - headers['Custom-header-' + str(i)] = 'a' * 8000 + headers[f'Custom-header-{i}'] = 'a' * 8000 assert self.get(headers=headers)['status'] == expect @@ -229,15 +229,12 @@ Connection: close sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sock.connect(addr) - req = ( - """GET / HTTP/1.1 + req = f"""GET / HTTP/1.1 Host: localhost -X-Length: %d +X-Length: {data_len} Connection: close """ - % data_len - ) sock.sendall(req.encode()) @@ -259,10 +256,10 @@ Connection: close data_len = 1048576 if len(values) == 0 else 10 * max(values) - addr = temp_dir + '/sock' + addr = f'{temp_dir}/sock' assert 'success' in self.conf( - {"unix:" + addr: {'application': 'body_generate'}}, 'listeners' + {f'unix:{addr}': {'application': 'body_generate'}}, 'listeners' ) assert 'success' in self.conf({'http': {'send_timeout': 1}}, 'settings') @@ -396,3 +393,157 @@ Connection: close assert bool(resp), 'response from application 4' assert resp['status'] == 200, 'status 4' assert resp['body'] == body, 'body 4' + + def test_settings_log_route(self): + def count_fallbacks(): + return len(self.findall(r'"fallback" taken')) + + def check_record(template): + assert self.search_in_log(template) is not None + + def check_no_record(template): + assert self.search_in_log(template) is None + + def template_req_line(url): + return rf'\[notice\].*http request line "GET {url} HTTP/1\.1"' + + def template_selected(route): + return rf'\[notice\].*"{route}" selected' + + def template_discarded(route): + return rf'\[info\].*"{route}" discarded' + + def wait_for_request_log(status, uri, route): + assert self.get(url=uri)['status'] == status + assert self.wait_for_record(template_req_line(uri)) is not None + assert self.wait_for_record(template_selected(route)) is not None + + # routes array + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes"}}, + "routes": [ + { + "match": { + "uri": "/zero", + }, + "action": {"return": 200}, + }, + { + "action": {"return": 201}, + }, + ], + "applications": {}, + "settings": {"http": {"log_route": True}}, + } + ) + + wait_for_request_log(200, '/zero', 'routes/0') + check_no_record(r'discarded') + + wait_for_request_log(201, '/one', 'routes/1') + check_record(template_discarded('routes/0')) + + # routes object + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes/main"}}, + "routes": { + "main": [ + { + "match": { + "uri": "/named_route", + }, + "action": {"return": 200}, + }, + { + "action": {"return": 201}, + }, + ] + }, + "applications": {}, + "settings": {"http": {"log_route": True}}, + } + ) + + wait_for_request_log(200, '/named_route', 'routes/main/0') + check_no_record(template_discarded('routes/main')) + + wait_for_request_log(201, '/unnamed_route', 'routes/main/1') + check_record(template_discarded('routes/main/0')) + + # routes sequence + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes/first"}}, + "routes": { + "first": [ + { + "action": {"pass": "routes/second"}, + }, + ], + "second": [ + { + "action": {"return": 200}, + }, + ], + }, + "applications": {}, + "settings": {"http": {"log_route": True}}, + } + ) + + wait_for_request_log(200, '/sequence', 'routes/second/0') + check_record(template_selected('routes/first/0')) + + # fallback + + assert 'success' in self.conf( + { + "listeners": {"*:7080": {"pass": "routes/fall"}}, + "routes": { + "fall": [ + { + "action": { + "share": "/blah", + "fallback": {"pass": "routes/fall2"}, + }, + }, + ], + "fall2": [ + { + "action": {"return": 200}, + }, + ], + }, + "applications": {}, + "settings": {"http": {"log_route": True}}, + } + ) + + wait_for_request_log(200, '/', 'routes/fall2/0') + assert count_fallbacks() == 1 + check_record(template_selected('routes/fall/0')) + + assert self.head()['status'] == 200 + assert count_fallbacks() == 2 + + # disable log + + assert 'success' in self.conf({"log_route": False}, 'settings/http') + + url = '/disable_logging' + assert self.get(url=url)['status'] == 200 + + time.sleep(1) + + check_no_record(template_req_line(url)) + + # total + + assert len(self.findall(r'\[notice\].*http request line')) == 7 + assert len(self.findall(r'\[notice\].*selected')) == 10 + assert len(self.findall(r'\[info\].*discarded')) == 2 diff --git a/test/test_static.py b/test/test_static.py index 9013b5c0..f7eade7c 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -11,13 +11,13 @@ class TestStatic(TestApplicationProto): prerequisites = {} def setup_method(self): - os.makedirs(option.temp_dir + '/assets/dir') - with open(option.temp_dir + '/assets/index.html', 'w') as index, open( - option.temp_dir + '/assets/README', 'w' + os.makedirs(f'{option.temp_dir}/assets/dir') + with open(f'{option.temp_dir}/assets/index.html', 'w') as index, open( + f'{option.temp_dir}/assets/README', 'w' ) as readme, open( - option.temp_dir + '/assets/log.log', 'w' + f'{option.temp_dir}/assets/log.log', 'w' ) as log, open( - option.temp_dir + '/assets/dir/file', 'w' + f'{option.temp_dir}/assets/dir/file', 'w' ) as file: index.write('0123456789') readme.write('readme') @@ -28,7 +28,7 @@ class TestStatic(TestApplicationProto): { "listeners": {"*:7080": {"pass": "routes"}}, "routes": [ - {"action": {"share": option.temp_dir + "/assets$uri"}} + {"action": {"share": f'{option.temp_dir}/assets$uri'}} ], "settings": { "http": { @@ -40,10 +40,10 @@ class TestStatic(TestApplicationProto): } ) - def test_static_index(self): + def test_static_index(self, temp_dir): def set_index(index): assert 'success' in self.conf( - {"share": option.temp_dir + "/assets$uri", "index": index}, + {"share": f'{temp_dir}/assets$uri', "index": index}, 'routes/0/action', ), 'configure index' @@ -72,12 +72,12 @@ class TestStatic(TestApplicationProto): resp['headers']['Content-Type'] == 'text/html' ), 'index not found 2 Content-Type' - def test_static_index_invalid(self, skip_alert): + def test_static_index_invalid(self, skip_alert, temp_dir): skip_alert(r'failed to apply new conf') def check_index(index): assert 'error' in self.conf( - {"share": option.temp_dir + "/assets$uri", "index": index}, + {"share": f'{temp_dir}/assets$uri', "index": index}, 'routes/0/action', ) @@ -86,7 +86,7 @@ class TestStatic(TestApplicationProto): def test_static_large_file(self, temp_dir): file_size = 32 * 1024 * 1024 - with open(temp_dir + '/assets/large', 'wb') as f: + with open(f'{temp_dir}/assets/large', 'wb') as f: f.seek(file_size - 1) f.write(b'\0') @@ -102,7 +102,7 @@ class TestStatic(TestApplicationProto): assert etag != etag_2, 'different ETag' assert etag == self.get(url='/')['headers']['ETag'], 'same ETag' - with open(temp_dir + '/assets/index.html', 'w') as f: + with open(f'{temp_dir}/assets/index.html', 'w') as f: f.write('blah') assert etag != self.get(url='/')['headers']['ETag'], 'new ETag' @@ -114,19 +114,21 @@ class TestStatic(TestApplicationProto): assert 'Content-Type' not in resp['headers'], 'redirect Content-Type' def test_static_space_in_name(self, temp_dir): + assets_dir = f'{temp_dir}/assets' + os.rename( - temp_dir + '/assets/dir/file', - temp_dir + '/assets/dir/fi le', + f'{assets_dir}/dir/file', + f'{assets_dir}/dir/fi le', ) - assert waitforfiles(temp_dir + '/assets/dir/fi le') + assert waitforfiles(f'{assets_dir}/dir/fi le') assert self.get(url='/dir/fi le')['body'] == 'blah', 'file name' - os.rename(temp_dir + '/assets/dir', temp_dir + '/assets/di r') - assert waitforfiles(temp_dir + '/assets/di r/fi le') + os.rename(f'{assets_dir}/dir', f'{assets_dir}/di r') + assert waitforfiles(f'{assets_dir}/di r/fi le') assert self.get(url='/di r/fi le')['body'] == 'blah', 'dir name' - os.rename(temp_dir + '/assets/di r', temp_dir + '/assets/ di r ') - assert waitforfiles(temp_dir + '/assets/ di r /fi le') + os.rename(f'{assets_dir}/di r', f'{assets_dir}/ di r ') + assert waitforfiles(f'{assets_dir}/ di r /fi le') assert ( self.get(url='/ di r /fi le')['body'] == 'blah' ), 'dir name enclosing' @@ -147,16 +149,16 @@ class TestStatic(TestApplicationProto): ), 'encoded 2' os.rename( - temp_dir + '/assets/ di r /fi le', - temp_dir + '/assets/ di r / fi le ', + f'{assets_dir}/ di r /fi le', + f'{assets_dir}/ di r / fi le ', ) - assert waitforfiles(temp_dir + '/assets/ di r / fi le ') + assert waitforfiles(f'{assets_dir}/ di r / fi le ') assert ( self.get(url='/%20di%20r%20/%20fi%20le%20')['body'] == 'blah' ), 'file name enclosing' try: - open(temp_dir + '/ф а', 'a').close() + open(f'{temp_dir}/ф а', 'a').close() utf8 = True except KeyboardInterrupt: @@ -167,33 +169,33 @@ class TestStatic(TestApplicationProto): if utf8: os.rename( - temp_dir + '/assets/ di r / fi le ', - temp_dir + '/assets/ di r /фа йл', + f'{assets_dir}/ di r / fi le ', + f'{assets_dir}/ di r /фа йл', ) - assert waitforfiles(temp_dir + '/assets/ di r /фа йл') + assert waitforfiles(f'{assets_dir}/ di r /фа йл') assert ( self.get(url='/ di r /фа йл')['body'] == 'blah' ), 'file name 2' os.rename( - temp_dir + '/assets/ di r ', - temp_dir + '/assets/ди ректория', + f'{assets_dir}/ di r ', + f'{assets_dir}/ди ректория', ) - assert waitforfiles(temp_dir + '/assets/ди ректория/фа йл') + assert waitforfiles(f'{assets_dir}/ди ректория/фа йл') assert ( self.get(url='/ди ректория/фа йл')['body'] == 'blah' ), 'dir name 2' def test_static_unix_socket(self, temp_dir): sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) - sock.bind(temp_dir + '/assets/unix_socket') + sock.bind(f'{temp_dir}/assets/unix_socket') assert self.get(url='/unix_socket')['status'] == 404, 'socket' sock.close() def test_static_unix_fifo(self, temp_dir): - os.mkfifo(temp_dir + '/assets/fifo') + os.mkfifo(f'{temp_dir}/assets/fifo') assert self.get(url='/fifo')['status'] == 404, 'fifo' @@ -346,5 +348,5 @@ Content-Length: 6\r raw_resp=True, raw=True, sock_type='unix', - addr=temp_dir + '/control.unit.sock', + addr=f'{temp_dir}/control.unit.sock', ), 'mime_types invalid' diff --git a/test/test_static_chroot.py b/test/test_static_chroot.py index e33a181c..c5a35d82 100644 --- a/test/test_static_chroot.py +++ b/test/test_static_chroot.py @@ -3,6 +3,7 @@ from pathlib import Path import pytest from unit.applications.proto import TestApplicationProto +from unit.option import option class TestStaticChroot(TestApplicationProto): @@ -10,22 +11,22 @@ class TestStaticChroot(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - os.makedirs(temp_dir + '/assets/dir') - Path(temp_dir + '/assets/index.html').write_text('0123456789') - Path(temp_dir + '/assets/dir/file').write_text('blah') + os.makedirs(f'{temp_dir}/assets/dir') + Path(f'{temp_dir}/assets/index.html').write_text('0123456789') + Path(f'{temp_dir}/assets/dir/file').write_text('blah') - self.test_path = '/' + os.path.relpath(Path(__file__)) + self.test_path = f'/{os.path.relpath(Path(__file__))}' self._load_conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], } ) - def update_action(self, share, chroot): + def update_action(self, chroot, share=f'{option.temp_dir}/assets$uri'): return self.conf( - {"share": share, "chroot": chroot}, + {'chroot': chroot, 'share': share}, 'routes/0/action', ) @@ -38,9 +39,7 @@ class TestStaticChroot(TestApplicationProto): assert self.get(url='/dir/file')['status'] == 200, 'default chroot' assert self.get(url='/index.html')['status'] == 200, 'default chroot 2' - assert 'success' in self.update_action( - temp_dir + "/assets$uri", temp_dir + "/assets/dir" - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/dir') assert self.get(url='/dir/file')['status'] == 200, 'chroot' assert self.get(url='/index.html')['status'] == 403, 'chroot 403 2' @@ -48,101 +47,89 @@ class TestStaticChroot(TestApplicationProto): def test_share_chroot_array(self, temp_dir): assert 'success' in self.update_action( - ["/blah", temp_dir + "/assets$uri"], temp_dir + "/assets/dir" + f'{temp_dir}/assets/dir', ["/blah", f'{temp_dir}/assets$uri'] ) assert self.get(url='/dir/file')['status'] == 200, 'share array' assert 'success' in self.update_action( - ["/blah", temp_dir + '/assets$uri'], temp_dir + '/assets/$host' + f'{temp_dir}/assets/$host', + ['/blah', f'{temp_dir}/assets$uri'], ) assert self.get_custom('/dir/file', 'dir') == 200, 'array variable' assert 'success' in self.update_action( - ["/blah", "/blah2"], temp_dir + "/assets/dir" + f'{temp_dir}/assets/dir', ['/blah', '/blah2'] ) assert self.get()['status'] != 200, 'share array bad' def test_static_chroot_permission(self, is_su, temp_dir): if is_su: - pytest.skip('does\'t work under root') + pytest.skip("does't work under root") - os.chmod(temp_dir + '/assets/dir', 0o100) + os.chmod(f'{temp_dir}/assets/dir', 0o100) assert 'success' in self.update_action( - temp_dir + "/assets$uri", temp_dir + "/assets/dir" + f'{temp_dir}/assets/dir' ), 'configure chroot' assert self.get(url='/dir/file')['status'] == 200, 'chroot' def test_static_chroot_empty(self, temp_dir): - assert 'success' in self.update_action(temp_dir + "/assets$uri", "") + assert 'success' in self.update_action('') assert self.get(url='/dir/file')['status'] == 200, 'empty absolute' - assert 'success' in self.update_action(".$uri", "") + assert 'success' in self.update_action("", ".$uri") assert self.get(url=self.test_path)['status'] == 200, 'empty relative' def test_static_chroot_relative(self, is_su, temp_dir): if is_su: - pytest.skip('does\'t work under root') + pytest.skip("Does't work under root.") - assert 'success' in self.update_action(temp_dir + "/assets$uri", ".") + assert 'success' in self.update_action('.') assert self.get(url='/dir/file')['status'] == 403, 'relative chroot' assert 'success' in self.conf({"share": ".$uri"}, 'routes/0/action') assert self.get(url=self.test_path)['status'] == 200, 'relative share' - assert 'success' in self.update_action(".$uri", ".") + assert 'success' in self.update_action(".", ".$uri") assert self.get(url=self.test_path)['status'] == 200, 'relative' def test_static_chroot_variables(self, temp_dir): - assert 'success' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/assets/$host' - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/$host') assert self.get_custom('/dir/file', 'dir') == 200 - assert 'success' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/assets/${host}' - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/${{host}}') assert self.get_custom('/dir/file', 'dir') == 200 def test_static_chroot_variables_buildin_start(self, temp_dir): assert 'success' in self.update_action( - temp_dir + '/assets/dir/$host', '$uri/assets/dir' + '$uri/assets/dir', + f'{temp_dir}/assets/dir/$host', ) assert self.get_custom(temp_dir, 'file') == 200 def test_static_chroot_variables_buildin_mid(self, temp_dir): - assert 'success' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/$host/dir' - ) + assert 'success' in self.update_action(f'{temp_dir}/$host/dir') assert self.get_custom('/dir/file', 'assets') == 200 def test_static_chroot_variables_buildin_end(self, temp_dir): - assert 'success' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/assets/$host' - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/$host') assert self.get_custom('/dir/file', 'dir') == 200 def test_static_chroot_slash(self, temp_dir): - assert 'success' in self.update_action( - temp_dir + "/assets$uri", temp_dir + "/assets/dir/" - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/dir/') assert self.get(url='/dir/file')['status'] == 200, 'slash end' assert self.get(url='/dirxfile')['status'] == 403, 'slash end bad' - assert 'success' in self.update_action( - temp_dir + "/assets$uri", temp_dir + "/assets/dir" - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/dir') assert self.get(url='/dir/file')['status'] == 200, 'no slash end' - assert 'success' in self.update_action( - temp_dir + "/assets$uri", temp_dir + "/assets/dir/" - ) + assert 'success' in self.update_action(f'{temp_dir}/assets/dir/') assert self.get(url='/dir/file')['status'] == 200, 'slash end 2' assert self.get(url='/dirxfile')['status'] == 403, 'slash end 2 bad' assert 'success' in self.update_action( - temp_dir + "///assets/////$uri", temp_dir + "//assets////dir///" + f'{temp_dir}//assets////dir///', f'{temp_dir}///assets/////$uri' ) assert self.get(url='/dir/file')['status'] == 200, 'multiple slashes' @@ -160,9 +147,5 @@ class TestStaticChroot(TestApplicationProto): 'routes/0/action', ), 'configure mount error' - assert 'error' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/assets/d$r$uri' - ) - assert 'error' in self.update_action( - temp_dir + '/assets$uri', temp_dir + '/assets/$$uri' - ) + assert 'error' in self.update_action(f'{temp_dir}/assets/d$r$uri') + assert 'error' in self.update_action(f'{temp_dir}/assets/$$uri') diff --git a/test/test_static_fallback.py b/test/test_static_fallback.py index 1f1a1df7..75012bbb 100644 --- a/test/test_static_fallback.py +++ b/test/test_static_fallback.py @@ -10,11 +10,12 @@ class TestStaticFallback(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - os.makedirs(temp_dir + '/assets/dir') - Path(temp_dir + '/assets/index.html').write_text('0123456789') + assets_dir = f'{temp_dir}/assets' + os.makedirs(f'{assets_dir}/dir') + Path(f'{assets_dir}/index.html').write_text('0123456789') - os.makedirs(temp_dir + '/assets/403') - os.chmod(temp_dir + '/assets/403', 0o000) + os.makedirs(f'{assets_dir}/403') + os.chmod(f'{assets_dir}/403', 0o000) self._load_conf( { @@ -22,7 +23,7 @@ class TestStaticFallback(TestApplicationProto): "*:7080": {"pass": "routes"}, "*:7081": {"pass": "routes"}, }, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{assets_dir}$uri'}}], "applications": {}, } ) @@ -30,7 +31,7 @@ class TestStaticFallback(TestApplicationProto): yield try: - os.chmod(temp_dir + '/assets/403', 0o777) + os.chmod(f'{assets_dir}/403', 0o777) except FileNotFoundError: pass @@ -49,7 +50,7 @@ class TestStaticFallback(TestApplicationProto): def test_static_fallback_valid_path(self, temp_dir): self.action_update( - {"share": temp_dir + "/assets$uri", "fallback": {"return": 200}} + {"share": f"{temp_dir}/assets$uri", "fallback": {"return": 200}} ) resp = self.get() assert resp['status'] == 200, 'fallback status' @@ -84,7 +85,7 @@ class TestStaticFallback(TestApplicationProto): self.action_update( { "share": "/blah", - "fallback": {"share": temp_dir + "/assets$uri"}, + "fallback": {"share": f"{temp_dir}/assets$uri"}, } ) diff --git a/test/test_static_mount.py b/test/test_static_mount.py index 91cf836c..406922b1 100644 --- a/test/test_static_mount.py +++ b/test/test_static_mount.py @@ -14,20 +14,20 @@ class TestStaticMount(TestApplicationProto): if not is_su: pytest.skip('requires root') - os.makedirs(temp_dir + '/assets/dir/mount') - os.makedirs(temp_dir + '/assets/dir/dir') - os.makedirs(temp_dir + '/assets/mount') - Path(temp_dir + '/assets/index.html').write_text('index') - Path(temp_dir + '/assets/dir/dir/file').write_text('file') - Path(temp_dir + '/assets/mount/index.html').write_text('mount') + os.makedirs(f'{temp_dir}/assets/dir/mount') + os.makedirs(f'{temp_dir}/assets/dir/dir') + os.makedirs(f'{temp_dir}/assets/mount') + Path(f'{temp_dir}/assets/index.html').write_text('index') + Path(f'{temp_dir}/assets/dir/dir/file').write_text('file') + Path(f'{temp_dir}/assets/mount/index.html').write_text('mount') try: subprocess.check_output( [ "mount", "--bind", - temp_dir + "/assets/mount", - temp_dir + "/assets/dir/mount", + f'{temp_dir}/assets/mount', + f'{temp_dir}/assets/dir/mount', ], stderr=subprocess.STDOUT, ) @@ -36,12 +36,12 @@ class TestStaticMount(TestApplicationProto): raise except subprocess.CalledProcessError: - pytest.fail('Can\'t run mount process.') + pytest.fail("Can't run mount process.") self._load_conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets/dir$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets/dir$uri'}}], } ) @@ -49,7 +49,7 @@ class TestStaticMount(TestApplicationProto): try: subprocess.check_output( - ["umount", "--lazy", temp_dir + "/assets/dir/mount"], + ["umount", "--lazy", f'{temp_dir}/assets/dir/mount'], stderr=subprocess.STDOUT, ) @@ -57,7 +57,7 @@ class TestStaticMount(TestApplicationProto): raise except subprocess.CalledProcessError: - pytest.fail('Can\'t run umount process.') + pytest.fail("Can't run umount process.") def test_static_mount(self, temp_dir, skip_alert): skip_alert(r'opening.*failed') @@ -67,14 +67,14 @@ class TestStaticMount(TestApplicationProto): assert resp['body'] == 'mount' assert 'success' in self.conf( - {"share": temp_dir + "/assets/dir$uri", "traverse_mounts": False}, + {"share": f'{temp_dir}/assets/dir$uri', "traverse_mounts": False}, 'routes/0/action', ), 'configure mount disable' assert self.get(url='/mount/')['status'] == 403 assert 'success' in self.conf( - {"share": temp_dir + "/assets/dir$uri", "traverse_mounts": True}, + {"share": f'{temp_dir}/assets/dir$uri', "traverse_mounts": True}, 'routes/0/action', ), 'configure mount enable' @@ -85,21 +85,21 @@ class TestStaticMount(TestApplicationProto): def test_static_mount_two_blocks(self, temp_dir, skip_alert): skip_alert(r'opening.*failed') - os.symlink(temp_dir + '/assets/dir', temp_dir + '/assets/link') + os.symlink(f'{temp_dir}/assets/dir', f'{temp_dir}/assets/link') assert 'success' in self.conf( [ { "match": {"method": "HEAD"}, "action": { - "share": temp_dir + "/assets/dir$uri", + "share": f'{temp_dir}/assets/dir$uri', "traverse_mounts": False, }, }, { "match": {"method": "GET"}, "action": { - "share": temp_dir + "/assets/dir$uri", + "share": f'{temp_dir}/assets/dir$uri', "traverse_mounts": True, }, }, @@ -115,8 +115,8 @@ class TestStaticMount(TestApplicationProto): assert 'success' in self.conf( { - "share": temp_dir + "/assets/dir$uri", - "chroot": temp_dir + "/assets", + "share": f'{temp_dir}/assets/dir$uri', + "chroot": f'{temp_dir}/assets', }, 'routes/0/action', ), 'configure chroot mount default' @@ -125,8 +125,8 @@ class TestStaticMount(TestApplicationProto): assert 'success' in self.conf( { - "share": temp_dir + "/assets/dir$uri", - "chroot": temp_dir + "/assets", + "share": f'{temp_dir}/assets/dir$uri', + "chroot": f'{temp_dir}/assets', "traverse_mounts": False, }, 'routes/0/action', diff --git a/test/test_static_share.py b/test/test_static_share.py index 5384866e..0166f1f0 100644 --- a/test/test_static_share.py +++ b/test/test_static_share.py @@ -10,16 +10,16 @@ class TestStaticShare(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - os.makedirs(temp_dir + '/assets/dir') - os.makedirs(temp_dir + '/assets/dir2') + os.makedirs(f'{temp_dir}/assets/dir') + os.makedirs(f'{temp_dir}/assets/dir2') - Path(temp_dir + '/assets/dir/file').write_text('1') - Path(temp_dir + '/assets/dir2/file2').write_text('2') + Path(f'{temp_dir}/assets/dir/file').write_text('1') + Path(f'{temp_dir}/assets/dir2/file2').write_text('2') assert 'success' in self.conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], "applications": {}, } ) @@ -31,7 +31,7 @@ class TestStaticShare(TestApplicationProto): assert self.get(url='/dir/file')['body'] == '1' assert self.get(url='/dir2/file2')['body'] == '2' - self.action_update({"share": [temp_dir + "/assets/dir$uri"]}) + self.action_update({"share": [f'{temp_dir}/assets/dir$uri']}) assert self.get(url='/file')['body'] == '1' assert self.get(url='/file2')['status'] == 404 @@ -39,8 +39,8 @@ class TestStaticShare(TestApplicationProto): self.action_update( { "share": [ - temp_dir + "/assets/dir$uri", - temp_dir + "/assets/dir2$uri", + f'{temp_dir}/assets/dir$uri', + f'{temp_dir}/assets/dir2$uri', ] } ) @@ -51,8 +51,8 @@ class TestStaticShare(TestApplicationProto): self.action_update( { "share": [ - temp_dir + "/assets/dir2$uri", - temp_dir + "/assets/dir3$uri", + f'{temp_dir}/assets/dir2$uri', + f'{temp_dir}/assets/dir3$uri', ] } ) diff --git a/test/test_static_symlink.py b/test/test_static_symlink.py index 24638e20..13d67bc7 100644 --- a/test/test_static_symlink.py +++ b/test/test_static_symlink.py @@ -10,21 +10,21 @@ class TestStaticSymlink(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - os.makedirs(temp_dir + '/assets/dir/dir') - Path(temp_dir + '/assets/index.html').write_text('0123456789') - Path(temp_dir + '/assets/dir/file').write_text('blah') + os.makedirs(f'{temp_dir}/assets/dir/dir') + Path(f'{temp_dir}/assets/index.html').write_text('0123456789') + Path(f'{temp_dir}/assets/dir/file').write_text('blah') self._load_conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], } ) def test_static_symlink(self, temp_dir, skip_alert): skip_alert(r'opening.*failed') - os.symlink(temp_dir + '/assets/dir', temp_dir + '/assets/link') + os.symlink(f'{temp_dir}/assets/dir', f'{temp_dir}/assets/link') assert self.get(url='/dir')['status'] == 301, 'dir' assert self.get(url='/dir/file')['status'] == 200, 'file' @@ -32,14 +32,14 @@ class TestStaticSymlink(TestApplicationProto): assert self.get(url='/link/file')['status'] == 200, 'symlink file' assert 'success' in self.conf( - {"share": temp_dir + "/assets$uri", "follow_symlinks": False}, + {"share": f'{temp_dir}/assets$uri', "follow_symlinks": False}, 'routes/0/action', ), 'configure symlink disable' assert self.get(url='/link/file')['status'] == 403, 'symlink disabled' assert 'success' in self.conf( - {"share": temp_dir + "/assets$uri", "follow_symlinks": True}, + {"share": f'{temp_dir}/assets$uri', "follow_symlinks": True}, 'routes/0/action', ), 'configure symlink enable' @@ -48,21 +48,21 @@ class TestStaticSymlink(TestApplicationProto): def test_static_symlink_two_blocks(self, temp_dir, skip_alert): skip_alert(r'opening.*failed') - os.symlink(temp_dir + '/assets/dir', temp_dir + '/assets/link') + os.symlink(f'{temp_dir}/assets/dir', f'{temp_dir}/assets/link') assert 'success' in self.conf( [ { "match": {"method": "HEAD"}, "action": { - "share": temp_dir + "/assets$uri", + "share": f'{temp_dir}/assets$uri', "follow_symlinks": False, }, }, { "match": {"method": "GET"}, "action": { - "share": temp_dir + "/assets$uri", + "share": f'{temp_dir}/assets$uri', "follow_symlinks": True, }, }, @@ -77,15 +77,15 @@ class TestStaticSymlink(TestApplicationProto): skip_alert(r'opening.*failed') os.symlink( - temp_dir + '/assets/dir/file', temp_dir + '/assets/dir/dir/link' + f'{temp_dir}/assets/dir/file', f'{temp_dir}/assets/dir/dir/link' ) assert self.get(url='/dir/dir/link')['status'] == 200, 'default chroot' assert 'success' in self.conf( { - "share": temp_dir + "/assets$uri", - "chroot": temp_dir + "/assets/dir/dir", + "share": f'{temp_dir}/assets$uri', + "chroot": f'{temp_dir}/assets/dir/dir', }, 'routes/0/action', ), 'configure chroot' diff --git a/test/test_static_types.py b/test/test_static_types.py index 0e86517b..28ab28e6 100644 --- a/test/test_static_types.py +++ b/test/test_static_types.py @@ -9,11 +9,11 @@ class TestStaticTypes(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - Path(temp_dir + '/assets').mkdir() + Path(f'{temp_dir}/assets').mkdir() for ext in ['.xml', '.mp4', '.php', '', '.txt', '.html', '.png']: - Path(temp_dir + '/assets/file' + ext).write_text(ext) + Path(f'{temp_dir}/assets/file{ext}').write_text(ext) - Path(temp_dir + '/assets/index.html').write_text('index') + Path(f'{temp_dir}/assets/index.html').write_text('index') self._load_conf( { @@ -21,7 +21,7 @@ class TestStaticTypes(TestApplicationProto): "*:7080": {"pass": "routes"}, "*:7081": {"pass": "routes"}, }, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], "applications": {}, } ) @@ -35,39 +35,39 @@ class TestStaticTypes(TestApplicationProto): assert resp['body'] == body, 'body' def test_static_types_basic(self, temp_dir): - self.action_update({"share": temp_dir + "/assets$uri"}) + self.action_update({"share": f'{temp_dir}/assets$uri'}) self.check_body('/index.html', 'index') self.check_body('/file.xml', '.xml') self.action_update( - {"share": temp_dir + "/assets$uri", "types": "application/xml"} + {"share": f'{temp_dir}/assets$uri', "types": "application/xml"} ) self.check_body('/file.xml', '.xml') self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["application/xml"]} + {"share": f'{temp_dir}/assets$uri', "types": ["application/xml"]} ) self.check_body('/file.xml', '.xml') - self.action_update({"share": temp_dir + "/assets$uri", "types": [""]}) + self.action_update({"share": f'{temp_dir}/assets$uri', "types": [""]}) assert self.get(url='/file.xml')['status'] == 403, 'no mtype' def test_static_types_wildcard(self, temp_dir): self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["application/*"]} + {"share": f'{temp_dir}/assets$uri', "types": ["application/*"]} ) self.check_body('/file.xml', '.xml') assert self.get(url='/file.mp4')['status'] == 403, 'app * mtype mp4' self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["video/*"]} + {"share": f'{temp_dir}/assets$uri', "types": ["video/*"]} ) assert self.get(url='/file.xml')['status'] == 403, 'video * mtype xml' self.check_body('/file.mp4', '.mp4') def test_static_types_negation(self, temp_dir): self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["!application/xml"]} + {"share": f'{temp_dir}/assets$uri', "types": ["!application/xml"]} ) assert self.get(url='/file.xml')['status'] == 403, 'forbidden negation' self.check_body('/file.mp4', '.mp4') @@ -75,7 +75,7 @@ class TestStaticTypes(TestApplicationProto): # sorting negation self.action_update( { - "share": temp_dir + "/assets$uri", + "share": f'{temp_dir}/assets$uri', "types": ["!video/*", "image/png", "!image/jpg"], } ) @@ -86,7 +86,7 @@ class TestStaticTypes(TestApplicationProto): def test_static_types_regex(self, temp_dir): self.action_update( { - "share": temp_dir + "/assets$uri", + "share": f'{temp_dir}/assets$uri', "types": ["~text/(html|plain)"], } ) @@ -96,7 +96,7 @@ class TestStaticTypes(TestApplicationProto): def test_static_types_case(self, temp_dir): self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["!APpliCaTiOn/xMl"]} + {"share": f'{temp_dir}/assets$uri', "types": ["!APpliCaTiOn/xMl"]} ) self.check_body('/file.mp4', '.mp4') assert ( @@ -104,7 +104,7 @@ class TestStaticTypes(TestApplicationProto): ), 'mixed case xml negation' self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["vIdEo/mp4"]} + {"share": f'{temp_dir}/assets$uri', "types": ["vIdEo/mp4"]} ) assert self.get(url='/file.mp4')['status'] == 200, 'mixed case' assert ( @@ -112,7 +112,7 @@ class TestStaticTypes(TestApplicationProto): ), 'mixed case video negation' self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["vIdEo/*"]} + {"share": f'{temp_dir}/assets$uri', "types": ["vIdEo/*"]} ) self.check_body('/file.mp4', '.mp4') assert ( @@ -128,7 +128,7 @@ class TestStaticTypes(TestApplicationProto): }, { "action": { - "share": temp_dir + "/assets$uri", + "share": f'{temp_dir}/assets$uri', "types": ["!application/x-httpd-php"], "fallback": {"proxy": "http://127.0.0.1:7081"}, } @@ -142,7 +142,7 @@ class TestStaticTypes(TestApplicationProto): def test_static_types_index(self, temp_dir): self.action_update( - {"share": temp_dir + "/assets$uri", "types": "application/xml"} + {"share": f'{temp_dir}/assets$uri', "types": "application/xml"} ) self.check_body('/', 'index') self.check_body('/file.xml', '.xml') @@ -153,7 +153,7 @@ class TestStaticTypes(TestApplicationProto): self._load_conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], "applications": {}, "settings": { "http": { @@ -163,10 +163,10 @@ class TestStaticTypes(TestApplicationProto): } ) - self.action_update({"share": temp_dir + "/assets$uri", "types": [""]}) + self.action_update({"share": f'{temp_dir}/assets$uri', "types": [""]}) assert self.get(url='/file')['status'] == 403, 'forbidden custom mime' self.action_update( - {"share": temp_dir + "/assets$uri", "types": ["test/mime-type"]} + {"share": f'{temp_dir}/assets$uri', "types": ["test/mime-type"]} ) self.check_body('/file', '') diff --git a/test/test_static_variables.py b/test/test_static_variables.py index e7e1629c..370c3e6f 100644 --- a/test/test_static_variables.py +++ b/test/test_static_variables.py @@ -10,16 +10,16 @@ class TestStaticVariables(TestApplicationProto): @pytest.fixture(autouse=True) def setup_method_fixture(self, temp_dir): - os.makedirs(temp_dir + '/assets/dir') - os.makedirs(temp_dir + '/assets/d$r') - Path(temp_dir + '/assets/index.html').write_text('0123456789') - Path(temp_dir + '/assets/dir/file').write_text('file') - Path(temp_dir + '/assets/d$r/file').write_text('d$r') + os.makedirs(f'{temp_dir}/assets/dir') + os.makedirs(f'{temp_dir}/assets/d$r') + Path(f'{temp_dir}/assets/index.html').write_text('0123456789') + Path(f'{temp_dir}/assets/dir/file').write_text('file') + Path(f'{temp_dir}/assets/d$r/file').write_text('d$r') self._load_conf( { "listeners": {"*:7080": {"pass": "routes"}}, - "routes": [{"action": {"share": temp_dir + "/assets$uri"}}], + "routes": [{"action": {"share": f'{temp_dir}/assets$uri'}}], } ) @@ -27,37 +27,37 @@ class TestStaticVariables(TestApplicationProto): if isinstance(share, list): return self.conf(share, 'routes/0/action/share') - return self.conf('"' + share + '"', 'routes/0/action/share') + return self.conf(f'"{share}"', 'routes/0/action/share') def test_static_variables(self, temp_dir): assert self.get(url='/index.html')['status'] == 200 assert self.get(url='/d$r/file')['status'] == 200 assert 'success' in self.update_share('$uri') - assert self.get(url=temp_dir + '/assets/index.html')['status'] == 200 + assert self.get(url=f'{temp_dir}/assets/index.html')['status'] == 200 - assert 'success' in self.update_share(temp_dir + '/assets${uri}') + assert 'success' in self.update_share(f'{temp_dir}/assets${{uri}}') assert self.get(url='/index.html')['status'] == 200 def test_static_variables_array(self, temp_dir): assert 'success' in self.update_share( - [temp_dir + '/assets$uri', '$uri'] + [f'{temp_dir}/assets$uri', '$uri'] ) assert self.get(url='/dir/file')['status'] == 200 - assert self.get(url=temp_dir + '/assets/index.html')['status'] == 200 + assert self.get(url=f'{temp_dir}/assets/index.html')['status'] == 200 assert self.get(url='/blah')['status'] == 404 assert 'success' in self.conf( { - "share": [temp_dir + '/assets$uri', '$uri'], + "share": [f'{temp_dir}/assets$uri', '$uri'], "fallback": {"return": 201}, }, 'routes/0/action', ) assert self.get(url='/dir/file')['status'] == 200 - assert self.get(url=temp_dir + '/assets/index.html')['status'] == 200 + assert self.get(url=f'{temp_dir}/assets/index.html')['status'] == 200 assert self.get(url='/dir/blah')['status'] == 201 def test_static_variables_buildin_start(self, temp_dir): @@ -65,15 +65,15 @@ class TestStaticVariables(TestApplicationProto): assert self.get(url=temp_dir)['status'] == 200 def test_static_variables_buildin_mid(self, temp_dir): - assert 'success' in self.update_share(temp_dir + '$uri/index.html') + assert 'success' in self.update_share(f'{temp_dir}$uri/index.html') assert self.get(url='/assets')['status'] == 200 def test_static_variables_buildin_end(self): assert self.get(url='/index.html')['status'] == 200 def test_static_variables_invalid(self, temp_dir): - assert 'error' in self.update_share(temp_dir + '/assets/d$r$uri') - assert 'error' in self.update_share(temp_dir + '/assets/$$uri') + assert 'error' in self.update_share(f'{temp_dir}/assets/d$r$uri') + assert 'error' in self.update_share(f'{temp_dir}/assets/$$uri') assert 'error' in self.update_share( - [temp_dir + '/assets$uri', temp_dir + '/assets/dir', '$$uri'] + [f'{temp_dir}/assets$uri', f'{temp_dir}/assets/dir', '$$uri'] ) diff --git a/test/test_status.py b/test/test_status.py index 6c733474..d0901f42 100644 --- a/test/test_status.py +++ b/test/test_status.py @@ -18,11 +18,12 @@ class TestStatus(TestApplicationPython): } def app_default(self, name="empty", module="wsgi"): + name_dir = f'{option.test_dir}/python/{name}' return { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/" + name, - "working_directory": option.test_dir + "/python/" + name, + "path": name_dir, + "working_directory": name_dir, "module": module, } @@ -141,7 +142,7 @@ Connection: close assert apps == expert.sort() def check_application(name, running, starting, idle, active): - Status.get('/applications/' + name) == { + Status.get(f'/applications/{name}') == { 'processes': { 'running': running, 'starting': starting, diff --git a/test/test_tls.py b/test/test_tls.py index d4edcbd3..06c38d0b 100644 --- a/test/test_tls.py +++ b/test/test_tls.py @@ -17,19 +17,19 @@ class TestTLS(TestApplicationTLS): def add_tls(self, application='empty', cert='default', port=7080): assert 'success' in self.conf( { - "pass": "applications/" + application, + "pass": f"applications/{application}", "tls": {"certificate": cert}, }, - 'listeners/*:' + str(port), + f'listeners/*:{port}', ) def remove_tls(self, application='empty', port=7080): assert 'success' in self.conf( - {"pass": "applications/" + application}, 'listeners/*:' + str(port) + {"pass": f"applications/{application}"}, f'listeners/*:{port}' ) def req(self, name='localhost', subject=None, x509=False): - subj = subject if subject is not None else '/CN=' + name + '/' + subj = subject if subject is not None else f'/CN={name}/' subprocess.check_output( [ @@ -39,27 +39,27 @@ class TestTLS(TestApplicationTLS): '-subj', subj, '-config', - option.temp_dir + '/openssl.conf', + f'{option.temp_dir}/openssl.conf', '-out', - option.temp_dir + '/' + name + '.csr', + f'{option.temp_dir}/{name}.csr', '-keyout', - option.temp_dir + '/' + name + '.key', + f'{option.temp_dir}/{name}.key', ], stderr=subprocess.STDOUT, ) def generate_ca_conf(self): - with open(option.temp_dir + '/ca.conf', 'w') as f: + with open(f'{option.temp_dir}/ca.conf', 'w') as f: f.write( - """[ ca ] + f"""[ ca ] default_ca = myca [ myca ] -new_certs_dir = %(dir)s -database = %(database)s +new_certs_dir = {option.temp_dir} +database = {option.temp_dir}/certindex default_md = sha256 policy = myca_policy -serial = %(certserial)s +serial = {option.temp_dir}/certserial default_days = 1 x509_extensions = myca_extensions copy_extensions = copy @@ -69,20 +69,15 @@ commonName = optional [ myca_extensions ] basicConstraints = critical,CA:TRUE""" - % { - 'dir': option.temp_dir, - 'database': option.temp_dir + '/certindex', - 'certserial': option.temp_dir + '/certserial', - } ) - with open(option.temp_dir + '/certserial', 'w') as f: + with open(f'{option.temp_dir}/certserial', 'w') as f: f.write('1000') - with open(option.temp_dir + '/certindex', 'w') as f: + with open(f'{option.temp_dir}/certindex', 'w') as f: f.write('') - with open(option.temp_dir + '/certindex.attr', 'w') as f: + with open(f'{option.temp_dir}/certindex.attr', 'w') as f: f.write('') def ca(self, cert='root', out='localhost'): @@ -92,15 +87,15 @@ basicConstraints = critical,CA:TRUE""" 'ca', '-batch', '-config', - option.temp_dir + '/ca.conf', + f'{option.temp_dir}/ca.conf', '-keyfile', - option.temp_dir + '/' + cert + '.key', + f'{option.temp_dir}/{cert}.key', '-cert', - option.temp_dir + '/' + cert + '.crt', + f'{option.temp_dir}/{cert}.crt', '-in', - option.temp_dir + '/' + out + '.csr', + f'{option.temp_dir}/{out}.csr', '-out', - option.temp_dir + '/' + out + '.crt', + f'{option.temp_dir}/{out}.crt', ], stderr=subprocess.STDOUT, ) @@ -109,9 +104,7 @@ basicConstraints = critical,CA:TRUE""" self.context = ssl.create_default_context() self.context.check_hostname = False self.context.verify_mode = ssl.CERT_REQUIRED - self.context.load_verify_locations( - option.temp_dir + '/' + cert + '.crt' - ) + self.context.load_verify_locations(f'{option.temp_dir}/{cert}.crt') def test_tls_listener_option_add(self): self.load('empty') @@ -230,7 +223,7 @@ basicConstraints = critical,CA:TRUE""" '-noout', '-genkey', '-out', - temp_dir + '/ec.key', + f'{temp_dir}/ec.key', '-name', 'prime256v1', ], @@ -246,11 +239,11 @@ basicConstraints = critical,CA:TRUE""" '-subj', '/CN=ec/', '-config', - temp_dir + '/openssl.conf', + f'{temp_dir}/openssl.conf', '-key', - temp_dir + '/ec.key', + f'{temp_dir}/ec.key', '-out', - temp_dir + '/ec.crt', + f'{temp_dir}/ec.crt', ], stderr=subprocess.STDOUT, ) @@ -305,9 +298,9 @@ basicConstraints = critical,CA:TRUE""" self.ca(cert='root', out='int') self.ca(cert='int', out='end') - crt_path = temp_dir + '/end-int.crt' - end_path = temp_dir + '/end.crt' - int_path = temp_dir + '/int.crt' + crt_path = f'{temp_dir}/end-int.crt' + end_path = f'{temp_dir}/end.crt' + int_path = f'{temp_dir}/int.crt' with open(crt_path, 'wb') as crt, open(end_path, 'rb') as end, open( int_path, 'rb' @@ -400,22 +393,24 @@ basicConstraints = critical,CA:TRUE""" elif i == chain_length - 1: self.req('end') else: - self.req('int{}'.format(i)) + self.req(f'int{i}') for i in range(chain_length - 1): if i == 0: self.ca(cert='root', out='int1') elif i == chain_length - 2: - self.ca(cert='int{}'.format(chain_length - 2), out='end') + self.ca(cert=f'int{(chain_length - 2)}', out='end') else: - self.ca(cert='int{}'.format(i), out='int{}'.format(i + 1)) + self.ca(cert=f'int{i}', out=f'int{(i + 1)}') for i in range(chain_length - 1, 0, -1): - path = temp_dir + ( - '/end.crt' if i == chain_length - 1 else '/int{}.crt'.format(i) + path = ( + f'{temp_dir}/end.crt' + if i == chain_length - 1 + else f'{temp_dir}/int{i}.crt' ) - with open(temp_dir + '/all.crt', 'a') as chain, open(path) as cert: + with open(f'{temp_dir}/all.crt', 'a') as chain, open(path) as cert: chain.write(cert.read()) self.set_certificate_req_context() @@ -611,10 +606,10 @@ basicConstraints = critical,CA:TRUE""" subprocess.check_output(['kill', '-9', app_id]) - skip_alert(r'process %s exited on signal 9' % app_id) + skip_alert(fr'process {app_id} exited on signal 9') self.wait_for_record( - r' (?!' + app_id + r'#)(\d+)#\d+ "mirror" application started' + fr' (?!{app_id}#)(\d+)#\d+ "mirror" application started' ) resp = self.post_ssl(sock=sock, body='0123456789') @@ -673,7 +668,7 @@ basicConstraints = critical,CA:TRUE""" } ) assert res['status'] == 200, 'status ok' - assert res['body'] == filename + data + assert res['body'] == f'{filename}{data}' def test_tls_multi_listener(self): self.load('empty') diff --git a/test/test_tls_conf_command.py b/test/test_tls_conf_command.py index b414b5a0..605848ea 100644 --- a/test/test_tls_conf_command.py +++ b/test/test_tls_conf_command.py @@ -47,7 +47,7 @@ class TestTLSConfCommand(TestApplicationTLS): assert 'success' in self.conf( { "certificate": "default", - "conf_commands": {"protocol": '-' + protocol}, + "conf_commands": {"protocol": f'-{protocol}'}, }, 'listeners/*:7080/tls', ), 'protocol disabled' @@ -74,8 +74,8 @@ class TestTLSConfCommand(TestApplicationTLS): { "certificate": "default", "conf_commands": { - "protocol": '-' + protocol, - "cipherstring": cipher[1] + ":!" + cipher[0], + "protocol": f'-{protocol}', + "cipherstring": f"{cipher[1]}:!{cipher[0]}", }, }, 'listeners/*:7080/tls', diff --git a/test/test_tls_sni.py b/test/test_tls_sni.py index 44cc21e1..e918bb20 100644 --- a/test/test_tls_sni.py +++ b/test/test_tls_sni.py @@ -30,17 +30,17 @@ class TestTLSSNI(TestApplicationTLS): assert 'success' in self.conf({"pass": "routes"}, 'listeners/*:7080') def generate_ca_conf(self): - with open(option.temp_dir + '/ca.conf', 'w') as f: + with open(f'{option.temp_dir}/ca.conf', 'w') as f: f.write( - """[ ca ] + f"""[ ca ] default_ca = myca [ myca ] -new_certs_dir = %(dir)s -database = %(database)s +new_certs_dir = {option.temp_dir} +database = {option.temp_dir}/certindex default_md = sha256 policy = myca_policy -serial = %(certserial)s +serial = {option.temp_dir}/certserial default_days = 1 x509_extensions = myca_extensions copy_extensions = copy @@ -50,17 +50,12 @@ commonName = optional [ myca_extensions ] basicConstraints = critical,CA:TRUE""" - % { - 'dir': option.temp_dir, - 'database': option.temp_dir + '/certindex', - 'certserial': option.temp_dir + '/certserial', - } ) - with open(option.temp_dir + '/certserial', 'w') as f: + with open(f'{option.temp_dir}/certserial', 'w') as f: f.write('1000') - with open(option.temp_dir + '/certindex', 'w') as f: + with open(f'{option.temp_dir}/certindex', 'w') as f: f.write('') def config_bundles(self, bundles): @@ -68,11 +63,7 @@ basicConstraints = critical,CA:TRUE""" for b in bundles: self.openssl_conf(rewrite=True, alt_names=bundles[b]['alt_names']) - subj = ( - '/CN={}/'.format(bundles[b]['subj']) - if 'subj' in bundles[b] - else '/' - ) + subj = f'/CN={bundles[b]["subj"]}/' if 'subj' in bundles[b] else '/' subprocess.check_output( [ @@ -82,11 +73,11 @@ basicConstraints = critical,CA:TRUE""" '-subj', subj, '-config', - option.temp_dir + '/openssl.conf', + f'{option.temp_dir}/openssl.conf', '-out', - option.temp_dir + '/{}.csr'.format(b), + f'{option.temp_dir}/{b}.csr', '-keyout', - option.temp_dir + '/{}.key'.format(b), + f'{option.temp_dir}/{b}.key', ], stderr=subprocess.STDOUT, ) @@ -94,11 +85,7 @@ basicConstraints = critical,CA:TRUE""" self.generate_ca_conf() for b in bundles: - subj = ( - '/CN={}/'.format(bundles[b]['subj']) - if 'subj' in bundles[b] - else '/' - ) + subj = f'/CN={bundles[b]["subj"]}/' if 'subj' in bundles[b] else '/' subprocess.check_output( [ @@ -108,15 +95,15 @@ basicConstraints = critical,CA:TRUE""" '-subj', subj, '-config', - option.temp_dir + '/ca.conf', + f'{option.temp_dir}/ca.conf', '-keyfile', - option.temp_dir + '/root.key', + f'{option.temp_dir}/root.key', '-cert', - option.temp_dir + '/root.crt', + f'{option.temp_dir}/root.crt', '-in', - option.temp_dir + '/{}.csr'.format(b), + f'{option.temp_dir}/{b}.csr', '-out', - option.temp_dir + '/{}.crt'.format(b), + f'{option.temp_dir}/{b}.crt', ], stderr=subprocess.STDOUT, ) @@ -124,7 +111,7 @@ basicConstraints = critical,CA:TRUE""" self.context = ssl.create_default_context() self.context.check_hostname = False self.context.verify_mode = ssl.CERT_REQUIRED - self.context.load_verify_locations(option.temp_dir + '/root.crt') + self.context.load_verify_locations(f'{option.temp_dir}/root.crt') self.load_certs(bundles) @@ -132,7 +119,7 @@ basicConstraints = critical,CA:TRUE""" for bname, bvalue in bundles.items(): assert 'success' in self.certificate_load( bname, bname - ), 'certificate {} upload'.format(bvalue['subj']) + ), f'certificate {bvalue["subj"]} upload' def check_cert(self, host, expect): resp, sock = self.get_ssl( diff --git a/test/test_tls_tickets.py b/test/test_tls_tickets.py index 5399fae7..cca230f3 100644 --- a/test/test_tls_tickets.py +++ b/test/test_tls_tickets.py @@ -48,7 +48,7 @@ class TestTLSTicket(TestApplicationTLS): def set_tickets(self, tickets=True, port=7080): assert 'success' in self.conf( {"cache_size": 0, "tickets": tickets}, - 'listeners/*:' + str(port) + '/tls/session', + f'listeners/*:{port}/tls/session', ) def connect(self, ctx=None, session=None, port=7080): @@ -168,7 +168,7 @@ class TestTLSTicket(TestApplicationTLS): 'listeners/*:7080/tls/session/tickets/0' ), 'removed first ticket' assert 'success' in self.conf_post( - '"' + self.ticket + '"', 'listeners/*:7080/tls/session/tickets' + f'"{self.ticket}"', 'listeners/*:7080/tls/session/tickets' ), 'add new ticket to the end of array' sess, ctx, _ = self.connect() @@ -186,10 +186,10 @@ class TestTLSTicket(TestApplicationTLS): check_tickets({}) check_tickets('!?&^' * 16) - check_tickets(self.ticket[:-2] + '!' + self.ticket[3:]) + check_tickets(f'{self.ticket[:-2]}!{self.ticket[3:]}') check_tickets(self.ticket[:-1]) - check_tickets(self.ticket + 'b') - check_tickets(self.ticket + 'blah') + check_tickets(f'{self.ticket}b') + check_tickets(f'{self.ticket}blah') check_tickets([True, self.ticket, self.ticket2]) check_tickets([self.ticket, 'blah', self.ticket2]) check_tickets([self.ticket, self.ticket2, []]) diff --git a/test/test_unix_abstract.py b/test/test_unix_abstract.py index 195b0aa7..c562487b 100644 --- a/test/test_unix_abstract.py +++ b/test/test_unix_abstract.py @@ -13,14 +13,14 @@ class TestUnixAbstract(TestApplicationPython): def source(source): assert 'success' in self.conf( - '"' + source + '"', 'routes/0/match/source' + f'"{source}"', 'routes/0/match/source' ) assert 'success' in self.conf( { "listeners": { "127.0.0.1:7080": {"pass": "routes"}, - "unix:@" + addr[1:]: {"pass": "routes"}, + f"unix:@{addr[1:]}": {"pass": "routes"}, }, "routes": [ { @@ -61,6 +61,7 @@ class TestUnixAbstract(TestApplicationPython): headers={'Connection': 'close', 'X-Forwarded-For': xff}, )['body'] + client_ip_dir = f"{option.test_dir}/python/client_ip" assert 'success' in self.conf( { "listeners": { @@ -90,9 +91,8 @@ class TestUnixAbstract(TestApplicationPython): "client_ip": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/client_ip", - "working_directory": option.test_dir - + "/python/client_ip", + "path": client_ip_dir, + "working_directory": client_ip_dir, "module": "wsgi", } }, diff --git a/test/test_upstreams_rr.py b/test/test_upstreams_rr.py index 71af3f5d..324c93cb 100644 --- a/test/test_upstreams_rr.py +++ b/test/test_upstreams_rr.py @@ -247,6 +247,7 @@ Connection: close assert abs(r_two[0] - r_two[1]) <= self.cpu_count, 'dep two mix' def test_upstreams_rr_delay(self): + delayed_dir = f'{option.test_dir}/python/delayed' assert 'success' in self.conf( { "listeners": { @@ -276,9 +277,8 @@ Connection: close "delayed": { "type": self.get_application_type(), "processes": {"spare": 0}, - "path": option.test_dir + "/python/delayed", - "working_directory": option.test_dir - + "/python/delayed", + "path": delayed_dir, + "working_directory": delayed_dir, "module": "wsgi", } }, @@ -389,20 +389,20 @@ Connection: close assert abs(resps[0] - resps[1]) <= self.cpu_count, 'post' def test_upstreams_rr_unix(self, temp_dir): - addr_0 = temp_dir + '/sock_0' - addr_1 = temp_dir + '/sock_1' + addr_0 = f'{temp_dir}/sock_0' + addr_1 = f'{temp_dir}/sock_1' assert 'success' in self.conf( { "*:7080": {"pass": "upstreams/one"}, - "unix:" + addr_0: {"pass": "routes/one"}, - "unix:" + addr_1: {"pass": "routes/two"}, + f"unix:{addr_0}": {"pass": "routes/one"}, + f"unix:{addr_1}": {"pass": "routes/two"}, }, 'listeners', ), 'configure listeners unix' assert 'success' in self.conf( - {"unix:" + addr_0: {}, "unix:" + addr_1: {}}, + {f"unix:{addr_0}": {}, f"unix:{addr_1}": {}}, 'upstreams/one/servers', ), 'configure servers unix' diff --git a/test/test_usr1.py b/test/test_usr1.py index 3d190935..4bff0242 100644 --- a/test/test_usr1.py +++ b/test/test_usr1.py @@ -14,15 +14,15 @@ class TestUSR1(TestApplicationPython): log = 'access.log' log_new = 'new.log' - log_path = temp_dir + '/' + log + log_path = f'{temp_dir}/{log}' assert 'success' in self.conf( - '"' + log_path + '"', 'access_log' + f'"{log_path}"', 'access_log' ), 'access log configure' assert waitforfiles(log_path), 'open' - os.rename(log_path, temp_dir + '/' + log_new) + os.rename(log_path, f'{temp_dir}/{log_new}') assert self.get()['status'] == 200 @@ -48,8 +48,8 @@ class TestUSR1(TestApplicationPython): self.load('log_body') log_new = 'new.log' - log_path = temp_dir + '/unit.log' - log_path_new = temp_dir + '/' + log_new + log_path = f'{temp_dir}/unit.log' + log_path_new = f'{temp_dir}/{log_new}' os.rename(log_path, log_path_new) diff --git a/test/test_variables.py b/test/test_variables.py index ecce5e6d..545d61e9 100644 --- a/test/test_variables.py +++ b/test/test_variables.py @@ -19,7 +19,7 @@ class TestVariables(TestApplicationProto): def set_format(self, format): assert 'success' in self.conf( { - 'path': option.temp_dir + '/access.log', + 'path': f'{option.temp_dir}/access.log', 'format': format, }, 'access_log', @@ -36,7 +36,7 @@ class TestVariables(TestApplicationProto): def check_dollar(location, expect): assert 'success' in self.conf( - '"' + location + '"', + f'"{location}"', 'routes/0/action/location', ) assert self.get()['headers']['Location'] == expect @@ -93,7 +93,7 @@ Connection: close self.set_format('$request_uri') def check_request_uri(req_uri): - reg = r'^' + re.escape(req_uri) + r'$' + reg = fr'^{re.escape(req_uri)}$' assert self.search_in_log(reg) is None assert self.get(url=req_uri)['status'] == 200 @@ -109,7 +109,7 @@ Connection: close def check_uri(uri, expect=None): expect = uri if expect is None else expect - reg = r'^' + re.escape(expect) + r'$' + reg = fr'^{re.escape(expect)}$' assert self.search_in_log(reg) is None assert self.get(url=uri)['status'] == 200 @@ -125,7 +125,7 @@ Connection: close def check_host(host, expect=None): expect = host if expect is None else expect - reg = r'^' + re.escape(expect) + r'$' + reg = fr'^{re.escape(expect)}$' assert self.search_in_log(reg) is None assert ( @@ -196,7 +196,7 @@ Connection: close self.set_format('$method $header_referer') def check_referer(referer): - reg = r'^GET ' + re.escape(referer) + r'$' + reg = fr'^GET {re.escape(referer)}$' assert self.search_in_log(reg) is None assert ( @@ -219,7 +219,7 @@ Connection: close self.set_format('$method $header_user_agent') def check_user_agent(user_agent): - reg = r'^GET ' + re.escape(user_agent) + r'$' + reg = fr'^GET {re.escape(user_agent)}$' assert self.search_in_log(reg) is None assert ( @@ -240,7 +240,7 @@ Connection: close def test_variables_many(self): def check_vars(uri, expect): - reg = r'^' + re.escape(expect) + r'$' + reg = fr'^{re.escape(expect)}$' assert self.search_in_log(reg) is None assert self.get(url=uri)['status'] == 200 @@ -273,7 +273,7 @@ Connection: close def test_variables_dynamic_arguments(self): def check_arg(url, expect=None): expect = url if expect is None else expect - reg = r'^' + re.escape(expect) + r'$' + reg = fr'^{re.escape(expect)}$' assert self.search_in_log(reg) is None assert self.get(url=url)['status'] == 200 @@ -304,7 +304,7 @@ Connection: close def test_variables_dynamic_headers(self): def check_header(header, value): - reg = r'^' + value + r'$' + reg = fr'^{value}$' assert self.search_in_log(reg) is None assert ( @@ -368,7 +368,7 @@ Connection: close def check_variables(format): assert 'error' in self.conf( { - 'path': option.temp_dir + '/access.log', + 'path': f'{option.temp_dir}/access.log', 'format': format, }, 'access_log', diff --git a/test/unit/applications/lang/go.py b/test/unit/applications/lang/go.py index 14e76362..557753a4 100644 --- a/test/unit/applications/lang/go.py +++ b/test/unit/applications/lang/go.py @@ -14,23 +14,21 @@ class TestApplicationGo(TestApplicationProto): except subprocess.CalledProcessError: return None - temp_dir = option.temp_dir + '/go/' + temp_dir = f'{option.temp_dir}/go/' if not os.path.exists(temp_dir): os.mkdir(temp_dir) - cache_dir = option.cache_dir + '/go-build' + cache_dir = f'{option.cache_dir}/go-build' if not os.path.exists(cache_dir): os.mkdir(cache_dir) env = os.environ.copy() - env['GOPATH'] = option.current_dir + '/build/go' + env['GOPATH'] = f'{option.current_dir}/build/go' env['GOCACHE'] = cache_dir - shutil.copy2( - option.test_dir + '/go/' + script + '/' + name + '.go', temp_dir - ) + shutil.copy2(f'{option.test_dir}/go/{script}/{name}.go', temp_dir) if static: args = [ @@ -41,21 +39,21 @@ class TestApplicationGo(TestApplicationProto): '-ldflags', '-extldflags "-static"', '-o', - temp_dir + name, - temp_dir + name + '.go', + f'{temp_dir}{name}', + f'{temp_dir}{name}.go', ] else: args = [ 'go', 'build', '-o', - temp_dir + name, - temp_dir + name + '.go', + f'{temp_dir}{name}', + f'{temp_dir}{name}.go', ] - replace_path = option.current_dir + '/build/go/src/unit.nginx.org/go' + replace_path = f'{option.current_dir}/build/go/src/unit.nginx.org/go' - with open(temp_dir + 'go.mod', 'w') as f: + with open(f'{temp_dir}go.mod', 'w') as f: f.write( f"""module test/app require unit.nginx.org/go v0.0.0 @@ -64,7 +62,7 @@ replace unit.nginx.org/go => {replace_path} ) if option.detailed: - print("\n$ GOPATH=" + env['GOPATH'] + " " + " ".join(args)) + print(f'\n$ GOPATH={env["GOPATH"]} {" ".join(args)}') try: output = subprocess.check_output( @@ -82,18 +80,18 @@ replace unit.nginx.org/go => {replace_path} def load(self, script, name='app', **kwargs): static_build = False - wdir = option.test_dir + "/go/" + script - executable = option.temp_dir + "/go/" + name + wdir = f'{option.test_dir}/go/{script}' + executable = f'{option.temp_dir}/go/{name}' if 'isolation' in kwargs and 'rootfs' in kwargs['isolation']: wdir = "/go/" - executable = "/go/" + name + executable = f"/go/{name}" static_build = True TestApplicationGo.prepare_env(script, name, static=static_build) conf = { - "listeners": {"*:7080": {"pass": "applications/" + script}}, + "listeners": {"*:7080": {"pass": f"applications/{script}"}}, "applications": { script: { "type": "external", diff --git a/test/unit/applications/lang/java.py b/test/unit/applications/lang/java.py index c8936274..b6382cfe 100644 --- a/test/unit/applications/lang/java.py +++ b/test/unit/applications/lang/java.py @@ -12,10 +12,10 @@ class TestApplicationJava(TestApplicationProto): application_type = "java" def prepare_env(self, script): - app_path = option.temp_dir + '/java' - web_inf_path = app_path + '/WEB-INF/' - classes_path = web_inf_path + 'classes/' - script_path = option.test_dir + '/java/' + script + '/' + app_path = f'{option.temp_dir}/java' + web_inf_path = f'{app_path}/WEB-INF/' + classes_path = f'{web_inf_path}classes/' + script_path = f'{option.test_dir}/java/{script}/' if not os.path.isdir(app_path): os.makedirs(app_path) @@ -23,7 +23,7 @@ class TestApplicationJava(TestApplicationProto): src = [] for f in os.listdir(script_path): - file_path = script_path + f + file_path = f'{script_path}{f}' if f.endswith('.java'): src.append(file_path) @@ -36,7 +36,7 @@ class TestApplicationJava(TestApplicationProto): if f == 'WEB-INF': continue - shutil.copytree(file_path, app_path + '/' + f) + shutil.copytree(file_path, f'{app_path}/{f}') continue if f == 'web.xml': @@ -52,11 +52,11 @@ class TestApplicationJava(TestApplicationProto): os.makedirs(classes_path) classpath = ( - option.current_dir + '/build/tomcat-servlet-api-9.0.70.jar' + f'{option.current_dir}/build/tomcat-servlet-api-9.0.70.jar' ) ws_jars = glob.glob( - option.current_dir + '/build/websocket-api-java-*.jar' + f'{option.current_dir}/build/websocket-api-java*.jar' ) if not ws_jars: @@ -74,12 +74,12 @@ class TestApplicationJava(TestApplicationProto): '-d', classes_path, '-classpath', - classpath + ':' + ws_jars[0], + f'{classpath}:{ws_jars[0]}', ] javac.extend(src) if option.detailed: - print("\n$ " + " ".join(javac)) + print(f'\n$ {" ".join(javac)}') try: subprocess.check_output(javac, stderr=subprocess.STDOUT) @@ -88,26 +88,24 @@ class TestApplicationJava(TestApplicationProto): raise except subprocess.CalledProcessError: - pytest.fail('Can\'t run javac process.') + pytest.fail("Can't run javac process.") def load(self, script, **kwargs): self.prepare_env(script) + script_path = f'{option.test_dir}/java/{script}/' self._load_conf( { - "listeners": {"*:7080": {"pass": "applications/" + script}}, + "listeners": {"*:7080": {"pass": f"applications/{script}"}}, "applications": { script: { - "unit_jars": option.current_dir + '/build', + "unit_jars": f'{option.current_dir}/build', "type": self.get_application_type(), "processes": {"spare": 0}, - "working_directory": option.test_dir - + '/java/' - + script - + '/', - "webapp": option.temp_dir + '/java', + "working_directory": script_path, + "webapp": f'{option.temp_dir}/java', } }, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/lang/node.py b/test/unit/applications/lang/node.py index 5d05c70c..87d5a19c 100644 --- a/test/unit/applications/lang/node.py +++ b/test/unit/applications/lang/node.py @@ -13,16 +13,16 @@ class TestApplicationNode(TestApplicationProto): def prepare_env(self, script): # copy application shutil.copytree( - option.test_dir + '/node/' + script, option.temp_dir + '/node' + f'{option.test_dir}/node/{script}', f'{option.temp_dir}/node' ) # copy modules shutil.copytree( - option.current_dir + '/node/node_modules', - option.temp_dir + '/node/node_modules', + f'{option.current_dir}/node/node_modules', + f'{option.temp_dir}/node/node_modules', ) - public_dir(option.temp_dir + '/node') + public_dir(f'{option.temp_dir}/node') def load(self, script, name='app.js', **kwargs): self.prepare_env(script) @@ -43,17 +43,17 @@ class TestApplicationNode(TestApplicationProto): self._load_conf( { "listeners": { - "*:7080": {"pass": "applications/" + quote(script, '')} + "*:7080": {"pass": f"applications/{quote(script, '')}"} }, "applications": { script: { "type": "external", "processes": {"spare": 0}, - "working_directory": option.temp_dir + '/node', + "working_directory": f'{option.temp_dir}/node', "executable": '/usr/bin/env', "arguments": arguments, } }, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/lang/perl.py b/test/unit/applications/lang/perl.py index 58b867f0..19852363 100644 --- a/test/unit/applications/lang/perl.py +++ b/test/unit/applications/lang/perl.py @@ -6,19 +6,19 @@ class TestApplicationPerl(TestApplicationProto): application_type = "perl" def load(self, script, name='psgi.pl', **kwargs): - script_path = option.test_dir + '/perl/' + script + script_path = f'{option.test_dir}/perl/{script}' self._load_conf( { - "listeners": {"*:7080": {"pass": "applications/" + script}}, + "listeners": {"*:7080": {"pass": f"applications/{script}"}}, "applications": { script: { "type": self.get_application_type(), "processes": {"spare": 0}, "working_directory": script_path, - "script": script_path + '/' + name, + "script": f'{script_path}/{name}', } }, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/lang/php.py b/test/unit/applications/lang/php.py index 5319d2ca..1b94c3ae 100644 --- a/test/unit/applications/lang/php.py +++ b/test/unit/applications/lang/php.py @@ -9,18 +9,18 @@ class TestApplicationPHP(TestApplicationProto): application_type = "php" def load(self, script, index='index.php', **kwargs): - script_path = option.test_dir + '/php/' + script + script_path = f'{option.test_dir}/php/{script}' if kwargs.get('isolation') and kwargs['isolation'].get('rootfs'): rootfs = kwargs['isolation']['rootfs'] - if not os.path.exists(rootfs + '/app/php/'): - os.makedirs(rootfs + '/app/php/') + if not os.path.exists(f'{rootfs}/app/php/'): + os.makedirs(f'{rootfs}/app/php/') - if not os.path.exists(rootfs + '/app/php/' + script): - shutil.copytree(script_path, rootfs + '/app/php/' + script) + if not os.path.exists(f'{rootfs}/app/php/{script}'): + shutil.copytree(script_path, f'{rootfs}/app/php/{script}') - script_path = '/app/php/' + script + script_path = f'/app/php/{script}' app = { "type": self.get_application_type(), @@ -41,8 +41,8 @@ class TestApplicationPHP(TestApplicationProto): self._load_conf( { - "listeners": {"*:7080": {"pass": "applications/" + script}}, + "listeners": {"*:7080": {"pass": f"applications/{script}"}}, "applications": {script: app}, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/lang/python.py b/test/unit/applications/lang/python.py index 3768cf07..0bb69992 100644 --- a/test/unit/applications/lang/python.py +++ b/test/unit/applications/lang/python.py @@ -20,18 +20,18 @@ class TestApplicationPython(TestApplicationProto): if script[0] == '/': script_path = script else: - script_path = option.test_dir + '/python/' + script + script_path = f'{option.test_dir}/python/{script}' if kwargs.get('isolation') and kwargs['isolation'].get('rootfs'): rootfs = kwargs['isolation']['rootfs'] - if not os.path.exists(rootfs + '/app/python/'): - os.makedirs(rootfs + '/app/python/') + if not os.path.exists(f'{rootfs}/app/python/'): + os.makedirs(f'{rootfs}/app/python/') - if not os.path.exists(rootfs + '/app/python/' + name): - shutil.copytree(script_path, rootfs + '/app/python/' + name) + if not os.path.exists(f'{rootfs}/app/python/{name}'): + shutil.copytree(script_path, f'{rootfs}/app/python/{name}') - script_path = '/app/python/' + name + script_path = f'/app/python/{name}' app = { "type": self.get_application_type(), @@ -58,9 +58,9 @@ class TestApplicationPython(TestApplicationProto): self._load_conf( { "listeners": { - "*:7080": {"pass": "applications/" + quote(name, '')} + "*:7080": {"pass": f"applications/{quote(name, '')}"} }, "applications": {name: app}, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/lang/ruby.py b/test/unit/applications/lang/ruby.py index 824bfe7f..e0712fc6 100644 --- a/test/unit/applications/lang/ruby.py +++ b/test/unit/applications/lang/ruby.py @@ -10,22 +10,22 @@ class TestApplicationRuby(TestApplicationProto): def prepare_env(self, script): shutil.copytree( - option.test_dir + '/ruby/' + script, - option.temp_dir + '/ruby/' + script, + f'{option.test_dir}/ruby/{script}', + f'{option.temp_dir}/ruby/{script}', ) - public_dir(option.temp_dir + '/ruby/' + script) + public_dir(f'{option.temp_dir}/ruby/{script}') def load(self, script, name='config.ru', **kwargs): self.prepare_env(script) - script_path = option.temp_dir + '/ruby/' + script + script_path = f'{option.temp_dir}/ruby/{script}' app = { "type": self.get_application_type(), "processes": {"spare": 0}, "working_directory": script_path, - "script": script_path + '/' + name, + "script": f'{script_path}/{name}', } for key in [ @@ -36,8 +36,8 @@ class TestApplicationRuby(TestApplicationProto): self._load_conf( { - "listeners": {"*:7080": {"pass": "applications/" + script}}, + "listeners": {"*:7080": {"pass": f"applications/{script}"}}, "applications": {script: app}, }, - **kwargs + **kwargs, ) diff --git a/test/unit/applications/tls.py b/test/unit/applications/tls.py index 93400328..e5813312 100644 --- a/test/unit/applications/tls.py +++ b/test/unit/applications/tls.py @@ -22,13 +22,13 @@ class TestApplicationTLS(TestApplicationProto): '-x509', '-new', '-subj', - '/CN=' + name + '/', + f'/CN={name}/', '-config', - option.temp_dir + '/openssl.conf', + f'{option.temp_dir}/openssl.conf', '-out', - option.temp_dir + '/' + name + '.crt', + f'{option.temp_dir}/{name}.crt', '-keyout', - option.temp_dir + '/' + name + '.key', + f'{option.temp_dir}/{name}.key', ], stderr=subprocess.STDOUT, ) @@ -40,11 +40,11 @@ class TestApplicationTLS(TestApplicationProto): if key is None: key = crt - key_path = option.temp_dir + '/' + key + '.key' - crt_path = option.temp_dir + '/' + crt + '.crt' + key_path = f'{option.temp_dir}/{key}.key' + crt_path = f'{option.temp_dir}/{crt}.crt' with open(key_path, 'rb') as k, open(crt_path, 'rb') as c: - return self.conf(k.read() + c.read(), '/certificates/' + crt) + return self.conf(k.read() + c.read(), f'/certificates/{crt}') def get_ssl(self, **kwargs): return self.get(wrapper=self.context.wrap_socket, **kwargs) @@ -52,54 +52,50 @@ class TestApplicationTLS(TestApplicationProto): def post_ssl(self, **kwargs): return self.post(wrapper=self.context.wrap_socket, **kwargs) - def openssl_conf(self, rewrite=False, alt_names=[]): - conf_path = option.temp_dir + '/openssl.conf' + def openssl_conf(self, rewrite=False, alt_names=None): + alt_names = alt_names or [] + conf_path = f'{option.temp_dir}/openssl.conf' if not rewrite and os.path.exists(conf_path): return # Generates alt_names section with dns names - a_names = "[alt_names]\n" + a_names = '[alt_names]\n' for i, k in enumerate(alt_names, 1): k = k.split('|') if k[0] == 'IP': - a_names += "IP.%d = %s\n" % (i, k[1]) + a_names += f'IP.{i} = {k[1]}\n' else: - a_names += "DNS.%d = %s\n" % (i, k[0]) + a_names += f'DNS.{i} = {k[0]}\n' # Generates section for sign request extension - a_sec = """req_extensions = myca_req_extensions + a_sec = f'''req_extensions = myca_req_extensions [ myca_req_extensions ] subjectAltName = @alt_names -{a_names}""".format( - a_names=a_names - ) +{a_names}''' with open(conf_path, 'w') as f: f.write( - """[ req ] + f'''[ req ] default_bits = 2048 encrypt_key = no distinguished_name = req_distinguished_name -{a_sec} -[ req_distinguished_name ]""".format( - a_sec=a_sec if alt_names else "" - ) +{a_sec if alt_names else ""} +[ req_distinguished_name ]''' ) def load(self, script, name=None): if name is None: name = script - script_path = option.test_dir + '/python/' + script - + script_path = f'{option.test_dir}/python/{script}' self._load_conf( { - "listeners": {"*:7080": {"pass": "applications/" + name}}, + "listeners": {"*:7080": {"pass": f"applications/{name}"}}, "applications": { name: { "type": "python", diff --git a/test/unit/applications/websockets.py b/test/unit/applications/websockets.py index 15f212ff..a4b9287d 100644 --- a/test/unit/applications/websockets.py +++ b/test/unit/applications/websockets.py @@ -52,7 +52,7 @@ class TestApplicationWebsocket(TestApplicationProto): while True: rlist = select.select([sock], [], [], 60)[0] if not rlist: - pytest.fail('Can\'t read response from server.') + pytest.fail("Can't read response from server.") resp += sock.recv(4096).decode() @@ -77,7 +77,7 @@ class TestApplicationWebsocket(TestApplicationProto): # For all current cases if the "read_timeout" was changed # than test do not expect to get a response from server. if read_timeout == 60: - pytest.fail('Can\'t read response from server.') + pytest.fail("Can't read response from server.") break data += sock.recv(bytes - len(data)) diff --git a/test/unit/check/chroot.py b/test/unit/check/chroot.py index 40b75058..1b7aae90 100644 --- a/test/unit/check/chroot.py +++ b/test/unit/check/chroot.py @@ -12,7 +12,7 @@ def check_chroot(): resp = http.put( url='/config', sock_type='unix', - addr=option.temp_dir + '/control.unit.sock', + addr=f'{option.temp_dir}/control.unit.sock', body=json.dumps( { "listeners": {"*:7080": {"pass": "routes"}}, diff --git a/test/unit/check/isolation.py b/test/unit/check/isolation.py index 9bd835a3..4ebce893 100644 --- a/test/unit/check/isolation.py +++ b/test/unit/check/isolation.py @@ -27,8 +27,8 @@ def check_isolation(): "empty": { "type": "external", "processes": {"spare": 0}, - "working_directory": option.test_dir + "/go/empty", - "executable": option.temp_dir + "/go/app", + "working_directory": f"{option.test_dir}/go/empty", + "executable": f"{option.test_dir}/go/app", "isolation": {"namespaces": {"credential": True}}, }, }, @@ -41,8 +41,8 @@ def check_isolation(): "empty": { "type": "python", "processes": {"spare": 0}, - "path": option.test_dir + "/python/empty", - "working_directory": option.test_dir + "/python/empty", + "path": f"{option.test_dir}/python/empty", + "working_directory": f"{option.test_dir}/python/empty", "module": "wsgi", "isolation": {"namespaces": {"credential": True}}, } @@ -56,8 +56,8 @@ def check_isolation(): "phpinfo": { "type": "php", "processes": {"spare": 0}, - "root": option.test_dir + "/php/phpinfo", - "working_directory": option.test_dir + "/php/phpinfo", + "root": f"{option.test_dir}/php/phpinfo", + "working_directory": f"{option.test_dir}/php/phpinfo", "index": "index.php", "isolation": {"namespaces": {"credential": True}}, } @@ -73,8 +73,8 @@ def check_isolation(): "empty": { "type": "ruby", "processes": {"spare": 0}, - "working_directory": option.temp_dir + "/ruby/empty", - "script": option.temp_dir + "/ruby/empty/config.ru", + "working_directory": f"{option.temp_dir}/ruby/empty", + "script": f"{option.temp_dir}/ruby/empty/config.ru", "isolation": {"namespaces": {"credential": True}}, } }, @@ -87,11 +87,11 @@ def check_isolation(): "listeners": {"*:7080": {"pass": "applications/empty"}}, "applications": { "empty": { - "unit_jars": option.current_dir + "/build", + "unit_jars": f"{option.current_dir}/build", "type": "java", "processes": {"spare": 0}, - "working_directory": option.test_dir + "/java/empty/", - "webapp": option.temp_dir + "/java", + "working_directory": f"{option.temp_dir}/java/empty/", + "webapp": f"{option.temp_dir}/java", "isolation": {"namespaces": {"credential": True}}, } }, @@ -106,7 +106,7 @@ def check_isolation(): "basic": { "type": "external", "processes": {"spare": 0}, - "working_directory": option.temp_dir + "/node", + "working_directory": f"{option.temp_dir}/node", "executable": "app.js", "isolation": {"namespaces": {"credential": True}}, } @@ -120,8 +120,8 @@ def check_isolation(): "body_empty": { "type": "perl", "processes": {"spare": 0}, - "working_directory": option.test_dir + "/perl/body_empty", - "script": option.test_dir + "/perl/body_empty/psgi.pl", + "working_directory": f"{option.test_dir}/perl/body_empty", + "script": f"{option.test_dir}/perl/body_empty/psgi.pl", "isolation": {"namespaces": {"credential": True}}, } }, @@ -133,7 +133,7 @@ def check_isolation(): resp = http.put( url='/config', sock_type='unix', - addr=option.temp_dir + '/control.unit.sock', + addr=f'{option.temp_dir}/control.unit.sock', body=json.dumps(conf), ) diff --git a/test/unit/check/node.py b/test/unit/check/node.py index e053a749..dd59e7a4 100644 --- a/test/unit/check/node.py +++ b/test/unit/check/node.py @@ -3,7 +3,7 @@ import subprocess def check_node(current_dir): - if not os.path.exists(current_dir + '/node/node_modules'): + if not os.path.exists(f'{current_dir}/node/node_modules'): return None try: diff --git a/test/unit/check/unix_abstract.py b/test/unit/check/unix_abstract.py index 5d1f629e..aadde43a 100644 --- a/test/unit/check/unix_abstract.py +++ b/test/unit/check/unix_abstract.py @@ -12,7 +12,7 @@ def check_unix_abstract(): resp = http.put( url='/config', sock_type='unix', - addr=option.temp_dir + '/control.unit.sock', + addr=f'{option.temp_dir}/control.unit.sock', body=json.dumps( { "listeners": {"unix:@sock": {"pass": "routes"}}, diff --git a/test/unit/control.py b/test/unit/control.py index 99436ca0..61b6edf4 100644 --- a/test/unit/control.py +++ b/test/unit/control.py @@ -21,7 +21,7 @@ def args_handler(conf_func): url = args[1] if len(args) == 2 else url_default - url = url if url.startswith('/') else url_default + '/' + url + url = url if url.startswith('/') else f'{url_default}/{url}' arguments = (self, url) if conf is None else (self, conf, url) return json.loads(conf_func(*arguments)) @@ -50,7 +50,7 @@ class TestControl(TestHTTP): args = { 'url': url, 'sock_type': 'unix', - 'addr': option.temp_dir + '/control.unit.sock', + 'addr': f'{option.temp_dir}/control.unit.sock', } if conf is not None: diff --git a/test/unit/http.py b/test/unit/http.py index c48a720f..6a267e26 100644 --- a/test/unit/http.py +++ b/test/unit/http.py @@ -53,13 +53,13 @@ class TestHTTP: sock.connect(connect_args) except (ConnectionRefusedError, FileNotFoundError): sock.close() - pytest.fail('Client can\'t connect to the server.') + pytest.fail("Client can't connect to the server.") else: sock = kwargs['sock'] if 'raw' not in kwargs: - req = ' '.join([start_str, url, http]) + crlf + req = f'{start_str} {url} {http}{crlf}' if body != b'': if isinstance(body, str): @@ -75,10 +75,10 @@ class TestHTTP: for header, value in headers.items(): if isinstance(value, list): for v in value: - req += header + ': ' + str(v) + crlf + req += f'{header}: {v}{crlf}' else: - req += header + ': ' + str(value) + crlf + req += f'{header}: {value}{crlf}' req = (req + crlf).encode() + body @@ -151,12 +151,12 @@ class TestHTTP: len_log = len(log) if len_log > limit: log = log[:limit] - appendix = '(...logged %s of %s bytes)' % (limit, len_log) + appendix = f'(...logged {limit} of {len_log} bytes)' if isinstance(log, bytes): appendix = appendix.encode() - log = log + appendix + log = f'{log}{appendix}' return log @@ -188,7 +188,7 @@ class TestHTTP: # For all current cases if the "read_timeout" was changed # than test do not expect to get a response from server. if timeout == timeout_default: - pytest.fail('Can\'t read response from server.') + pytest.fail("Can't read response from server.") break try: @@ -263,7 +263,7 @@ class TestHTTP: size = int(chunks.pop(0), 16) except ValueError: - pytest.fail('Invalid chunk size %s' % str(size)) + pytest.fail(f'Invalid chunk size {size}') if size == 0: assert len(chunks) == 1, 'last zero size' @@ -310,7 +310,7 @@ class TestHTTP: def form_url_encode(self, fields): data = "&".join( - "%s=%s" % (name, value) for name, value in fields.items() + f'{name}={value}' for name, value in fields.items() ).encode() return data, 'application/x-www-form-urlencoded' @@ -341,21 +341,21 @@ class TestHTTP: else: pytest.fail('multipart requires a string or stream data') - body += ("--%s\r\nContent-Disposition: form-data; name=\"%s\"") % ( - boundary, - field, + body += ( + f'--{boundary}\r\nContent-Disposition: form-data;' + f'name="{field}"' ) if filename != '': - body += "; filename=\"%s\"" % filename + body += f'; filename="{filename}"' - body += "\r\n" + body += '\r\n' if datatype != '': - body += "Content-Type: %s\r\n" % datatype + body += f'Content-Type: {datatype}\r\n' - body += "\r\n%s\r\n" % data + body += f'\r\n{data}\r\n' - body += "--%s--\r\n" % boundary + body += f'--{boundary}--\r\n' - return body.encode(), "multipart/form-data; boundary=%s" % boundary + return body.encode(), f'multipart/form-data; boundary={boundary}' diff --git a/test/unit/log.py b/test/unit/log.py index 7263443d..f984d7a1 100644 --- a/test/unit/log.py +++ b/test/unit/log.py @@ -20,4 +20,4 @@ class Log: Log.pos[name] = pos def get_path(name=UNIT_LOG): - return Log.temp_dir + '/' + name + return f'{Log.temp_dir}/{name}' diff --git a/test/unit/utils.py b/test/unit/utils.py index d6590b97..985801e2 100644 --- a/test/unit/utils.py +++ b/test/unit/utils.py @@ -69,7 +69,7 @@ def waitforsocket(port): except KeyboardInterrupt: raise - pytest.fail('Can\'t connect to the 127.0.0.1:' + str(port)) + pytest.fail(f"Can't connect to the 127.0.0.1:{port}") def check_findmnt(): @@ -125,7 +125,7 @@ def getns(nstype): # read namespace id from symlink file: # it points to: '<nstype>:[<ns id>]' # # eg.: 'pid:[4026531836]' - nspath = '/proc/self/ns/' + nstype + nspath = f'/proc/self/ns/{nstype}' data = None if os.path.exists(nspath): diff --git a/tools/setup-unit b/tools/setup-unit index 79dab850..de1d4f5f 100755 --- a/tools/setup-unit +++ b/tools/setup-unit @@ -9,11 +9,13 @@ if test -n ${BASH_VERSION} && test "${BASH_VERSINFO[0]}" -eq 3; then - >&2 echo 'Your version of bash(1) isn't supported by this script.'; - >&2 echo "You're probably running on macOS. We recommend that you either"; - >&2 echo 'install a newer version of bash(1) or run this script with'; - >&2 echo 'another shell, such as zsh(1):'; - >&2 echo " $ zsh ${SUDO_USER:+sudo }$0 ..."; + >&2 cat <<__EOF__ ; +Your version of bash(1) isn't supported by this script. You're probably +running on macOS. We recommend that you either install a newer version +of bash(1) or run this script with another shell, such as zsh(1): + + $ ${SUDO_USER:+sudo }zsh $0 ... +__EOF__ exit 1; fi; @@ -23,6 +25,9 @@ set -Eefuo pipefail; test -v BASH_VERSION \ && shopt -s lastpipe; +test -v ZSH_VERSION \ +&& setopt sh_word_split; + export LC_ALL=C program_name="$0"; @@ -37,8 +42,8 @@ SYNOPSIS $program_name [-h] COMMAND [ARGS] Subcommands - +-- repo-config [-hn] [PKG-MANAGER OS-NAME OS-VERSION] - +-- welcome [-hn] + ├── repo-config [-hn] [PKG-MANAGER OS-NAME OS-VERSION] + └── welcome [-hn] DESCRIPTION This script simplifies installing and configuring an NGINX Unit server @@ -73,19 +78,19 @@ SYNOPSIS $program_name [-h] COMMAND [ARGS] Subcommands - +-- cmd [-h] - +-- ctl [-h] [-s SOCK] SUBCOMMAND [ARGS] - | +-- http [-h] [-c CURLOPT] METHOD PATH - | +-- insert [-h] PATH INDEX - +-- freeport [-h] - +-- json-ins [-hn] JSON INDEX - +-- os-probe [-h] - +-- ps [-h] [-t TYPE] - +-- repo-config [-hn] [PKG-MANAGER OS-NAME OS-VERSION] - +-- sock [-h] SUBCOMMAND [ARGS] - | +-- filter [-chs] - | +-- find [-h] - +-- welcome [-hn] + ├── cmd [-h] + ├── ctl [-h] [-s SOCK] SUBCOMMAND [ARGS] + │ ├── http [-h] [-c CURLOPT] METHOD PATH + │ └── insert [-h] PATH INDEX + ├── freeport [-h] + ├── json-ins [-hn] JSON INDEX + ├── os-probe [-h] + ├── ps [-h] [-t TYPE] + ├── repo-config [-hn] [PKG-MANAGER OS-NAME OS-VERSION] + ├── sock [-h] SUBCOMMAND [ARGS] + │ ├── filter [-chs] + │ └── find [-h] + └── welcome [-hn] DESCRIPTION This script simplifies installing and configuring @@ -205,8 +210,8 @@ SYNOPSIS $program_name ctl [-h] [-s SOCK] SUBCOMMAND [ARGS] Subcommands - +-- http [-h] [-c CURLOPT] METHOD PATH - +-- insert [-h] PATH INDEX + ├── http [-h] [-c CURLOPT] METHOD PATH + └── insert [-h] PATH INDEX DESCRIPTION Control a running unitd(8) instance through its control API socket. @@ -285,7 +290,11 @@ unit_ctl() err 'ctl: Missing subcommand.'; fi; - if test -v sock && echo $sock | grep '^ssh://' >/dev/null; then + if ! test -v sock; then + local sock="$(unit_sock_find)"; + fi; + + if echo $sock | grep '^ssh://' >/dev/null; then local remote="$(echo $sock | sed 's,\(ssh://[^/]*\).*,\1,')"; local sock="$(echo $sock | sed 's,ssh://[^/]*\(.*\),unix:\1,')"; fi; @@ -293,11 +302,11 @@ unit_ctl() case $1 in http) shift; - unit_ctl_http ${remote:+ ---r $remote} ${sock:+ ---s $sock} $@; + unit_ctl_http ${remote:+ ---r $remote} ---s "$sock" $@; ;; insert) shift; - unit_ctl_insert ${remote:+ ---r $remote} ${sock:+ ---s $sock} $@; + unit_ctl_insert ${remote:+ ---r $remote} ---s "$sock" $@; ;; *) err "ctl: $1: Unknown argument."; @@ -398,9 +407,6 @@ unit_ctl_http() -L "$local_sock:$remote_sock" "$remote"; sock="unix:$local_sock"; - - elif ! test -v sock; then - local sock="$(unit_sock_find)"; fi; curl $curl_options -X $method -d@- \ @@ -485,9 +491,6 @@ unit_ctl_insert() -L "$local_sock:$remote_sock" "$remote"; sock="unix:$local_sock"; - - elif ! test -v sock; then - local sock="$(unit_sock_find)"; fi; local old="$(mktemp ||:)"; @@ -555,15 +558,14 @@ unit_ctl_welcome() shift; done; - id -u \ - | xargs test 0 -ne \ - && err 'welcome: This script requires root privileges to run.'; - command -v curl >/dev/null \ || err 'welcome: curl(1) not found in PATH. It must be installed to run this script.'; www='/srv/www/unit/index.html'; if test -e "$www" && ! test -v force || ! test -w /srv; then + www="$HOME/srv/www/unit/index.html"; + fi; + if test -e "$www" && ! test -v force; then www="$(mktemp)"; mv "$www" "$www.html"; www="$www.html" @@ -594,14 +596,14 @@ unit_ctl_welcome() # Check unitd is not configured already. echo "$cmd" \ - | if grep '\--state' >/dev/null; then + | if grep '\--statedir' >/dev/null; then echo "$cmd" \ | sed 's/ --/\n--/g' \ - | grep '\--state' \ + | grep '\--statedir' \ | cut -d' ' -f2; else $cmd --help \ - | sed -n '/\--state/,+1p' \ + | sed -n '/\--statedir/,+1p' \ | grep 'default:' \ | sed 's/ *default: "\(.*\)"/\1/'; fi \ @@ -626,7 +628,7 @@ unit_ctl_welcome() dry_run_echo 'Create a file to serve:'; dry_run_eval "mkdir -p $(dirname $www);"; - dry_run_eval "cat >'$www'"' <<__EOF__; + dry_run_eval "tee '$www' >/dev/null"' <<__EOF__; <!DOCTYPE html> <html> <head> @@ -1177,10 +1179,6 @@ __EOF__"; command -v curl >/dev/null \ || err 'repo-config: curl(1) not found in PATH. It must be installed to run this script.'; - id -u \ - | xargs test 0 -ne \ - && err 'repo-config: This script requires root privileges to run.'; - echo 'This script sets up the NGINX Unit repository'; if ! test $# -ge 3; then @@ -1231,8 +1229,8 @@ SYNOPSIS $program_name sock [-h] SUBCOMMAND [ARGS] Subcommands - +-- filter [-ch] - +-- find [-h] + ├── filter [-ch] + └── find [-h] DESCRIPTION Print the control API socket address of running unitd(8) diff --git a/tools/unitc b/tools/unitc index 838f7ebf..9973e62d 100755 --- a/tools/unitc +++ b/tools/unitc @@ -43,7 +43,7 @@ while [ $# -gt 0 ]; do ;; *) - if [ -r $1 ]; then + if [ -f $1 ] && [ -r $1 ]; then CONF_FILES+=($1) elif [ "${1:0:1}" = "/" ] || [ "${1:0:4}" = "http" ] && [ "$URI" = "" ]; then URI=$1 @@ -121,9 +121,9 @@ if [ $REMOTE -eq 0 ]; then if [ -r /tmp/${0##*/}.$PID.env ]; then source /tmp/${0##*/}.$PID.env else - # Check we have unitd in $PATH (and all the other tools we will need) + # Check we have all the tools we will need (that we didn't already use) # - MISSING=$(hash unitd curl ps grep tr cut sed tail sleep 2>&1 | cut -f4 -d: | tr -d '\n') + MISSING=$(hash curl tr cut sed tail sleep 2>&1 | cut -f4 -d: | tr -d '\n') if [ "$MISSING" != "" ]; then echo "${0##*/}: ERROR: cannot find$MISSING: please install or add to \$PATH" exit 1 @@ -131,10 +131,10 @@ if [ $REMOTE -eq 0 ]; then # Get control address # - PARAMS=$(ps $PID | grep unitd | cut -f2- -dv | tr '[]' ' ' | cut -f4- -d ' ' | sed -e 's/ --/\n--/g') + PARAMS=$(ps $PID | grep unitd | cut -f2- -dv | tr '[]' ' ' | cut -f3- -d ' ' | sed -e 's/ --/\n--/g') CTRL_ADDR=$(echo "$PARAMS" | grep '\--control' | cut -f2 -d' ') if [ "$CTRL_ADDR" = "" ]; then - CTRL_ADDR=$(unitd --help | grep -A1 '\--control' | tail -1 | cut -f2 -d\") + CTRL_ADDR=$($(echo "$PARAMS" | grep unitd) --help | grep -A1 '\--control' | tail -1 | cut -f2 -d\") fi # Prepare for network or Unix socket addressing @@ -156,12 +156,12 @@ if [ $REMOTE -eq 0 ]; then # ERROR_LOG=$(echo "$PARAMS" | grep '\--log' | cut -f2 -d' ') if [ "$ERROR_LOG" = "" ]; then - ERROR_LOG=$(unitd --help | grep -A1 '\--log' | tail -1 | cut -f2 -d\") + ERROR_LOG=$($(echo "$PARAMS" | grep unitd) --help | grep -A1 '\--log' | tail -1 | cut -f2 -d\") fi # Cache the discovery for this unit PID (and cleanup any old files) # - rm /tmp/${0##*/}.* 2> /dev/null + rm -f /tmp/${0##*/}.* 2> /dev/null echo UNIT_CTRL=\"${UNIT_CTRL}\" > /tmp/${0##*/}.$PID.env echo ERROR_LOG=${ERROR_LOG} >> /tmp/${0##*/}.$PID.env fi @@ -221,11 +221,11 @@ if [ $CURL_STATUS -ne 0 ]; then echo "${0##*/}: Check that you have permission to access the Unit control socket, or try again with sudo(8)" else echo "${0##*/}: Trying to access $UNIT_CTRL$URI" - cat /tmp/${0##*/}.$$ && rm /tmp/${0##*/}.$$ + cat /tmp/${0##*/}.$$ && rm -f /tmp/${0##*/}.$$ fi exit 4 fi -rm /tmp/${0##*/}.$$ 2> /dev/null +rm -f /tmp/${0##*/}.$$ 2> /dev/null if [ $SHOW_LOG -gt 0 ] && [ $NOLOG -eq 0 ] && [ $QUIET -eq 0 ]; then echo -n "${0##*/}: Waiting for log..." @@ -1,5 +1,5 @@ # Copyright (C) NGINX, Inc. -NXT_VERSION=1.29.1 -NXT_VERNUM=12901 +NXT_VERSION=1.30.0 +NXT_VERNUM=13000 |