summaryrefslogtreecommitdiffhomepage
path: root/test/test_access_log.py (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2023-04-12HTTP: optimizing $request_line.Alejandro Colomar1-4/+4
Don't reconstruct a new string for the $request_line from the parsed method, target, and HTTP version, but rather keep a pointer to the original memory where the request line was received. This will be necessary for implementing URI rewrites, since we want to log the original request line, and not one constructed from the rewritten target. This implementation changes behavior (only for invalid requests) in the following way: Previous behavior was to log as many tokens from the request line as were parsed validly, thus: Request -> access log ; error log "GET / HTTP/1.1" -> "GET / HTTP/1.1" OK ; = "GET / HTTP/1.1" -> "GET / HTTP/1.1" [1] ; = "GET / HTTP/2.1" -> "GET / HTTP/2.1" OK ; = "GET / HTTP/1." -> "GET / HTTP/1." [2] ; "GET / HTTP/1. [null]" "GET / food" -> "GET / food" [2] ; "GET / food [null]" "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; = "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; = "GET food HTTP/1.1" -> "GET" ; "GET [null] [null]" "OPTIONS * HTTP/1.1" -> "OPTIONS" [3] ; "OPTIONS [null] [null]" "FOOBAR baz HTTP/1.1"-> "FOOBAR" ; "FOOBAR [null] [null]" "FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1" ; = "get / HTTP/1.1" -> "-" ; " [null] [null]" "" -> "-" ; " [null] [null]" This behavior was rather inconsistent. We have several options to go forward with this patch: - NGINX behavior. Log the entire request line, up to '\r' | '\n', even if it was invalid. This is the most informative alternative. However, RFC-complying requests will probably not send invalid requests. This information would be interesting to users where debugging requests constructed manually via netcat(1) or a similar tool, or maybe for debugging a client, are important. It might be interesting to support this in the future if our users are interested; for now, since this approach requires looping over invalid requests twice, that's an overhead that we better avoid. - Previous Unit behavior This is relatively fast (almost as fast as the next alternative, the one we chose), but the implementation is ugly, in that we need to perform the same operation in many places around the code. If we want performance, probably the next alternative is better; if we want to be informative, then the first one is better (maybe in combination with the third one too). - Chosen behavior Only logging request lines when the request is valid. For any invalid request, or even unsupported ones, the request line will be logged as "-". Thus: Request -> access log [4] "GET / HTTP/1.1" -> "GET / HTTP/1.1" OK "GET / HTTP/1.1" -> "GET / HTTP/1.1" [1] "GET / HTTP/2.1" -> "-" [3] "GET / HTTP/1." -> "-" "GET / food" -> "-" "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] "GET food HTTP/1.1" -> "-" "OPTIONS * HTTP/1.1" -> "-" "FOOBAR baz HTTP/1.1"-> "-" "FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1" "get / HTTP/1.1" -> "-" "" -> "-" This is less informative than previous behavior, but considering how inconsistent it was, and that RFC-complying agents will probably not send us such requests, we're ready to lose that information in the log. This is of course the fastest and simplest implementation we can get. We've chosen to implement this alternative in this patch. Since we modified the behavior, this patch also changes the affected tests. [1]: Multiple successive spaces as a token delimiter is allowed by the RFC, but it is discouraged, and considered a security risk. It is currently supported by Unit, but we will probably drop support for it in the future. [2]: Unit currently supports spaces in the request-target. This is a violation of the relevant RFC (linked below), and will be fixed in the future, and consider those targets as invalid, returning a 400 (Bad Request), and thus the log lines with the previous inconsistent behavior would be changed. [3]: Not yet supported. [4]: In the error log, regarding the "log_routes" conditional logging of the request line, we only need to log the request line if it was valid. It doesn't make sense to log "" or "-" in case that the request was invalid, since this is only useful for understanding decisions of the router. In this case, the access log is more appropriate, which shows that the request was invalid, and a 400 was returned. When the request line is valid, it is printed in the error log exactly as in the access log. Link: <https://datatracker.ietf.org/doc/html/rfc9112#section-3> Suggested-by: Liam Crilly <liam@nginx.com> Reviewed-by: Zhidao Hong <z.hong@f5.com> Cc: Timo Stark <t.stark@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Cc: Andrew Clayton <a.clayton@nginx.com> Cc: Artem Konev <a.konev@f5.com> Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-02-21Tests: switched to using f-strings.Andrei Zeliankou1-13/+12
Previously, it was necessary to support older versions of Python for compatibility. F-strings were released in Python 3.6. Python 3.5 was marked as unsupported by the end of 2020, so now it's possible to start using f-strings safely for better readability and performance.
2022-10-11Tests: reworked "test_variables.py".Andrei Zeliankou1-22/+0
Access log used for the variables testing instead of limited routing. Added missed test for $status variable. Some tests moved from "test_access_log.py" to "test_variables.py".
2022-09-07Tests: minor fixes.Andrei Zeliankou1-20/+2
2022-07-28Tests: added tests for the log format.Andrei Zeliankou1-0/+80
Also added tests for the following variables: $request_line, $time_local, $bytes_sent, and $status.
2021-10-18Tests: style.Andrei Zeliankou1-1/+0
2021-04-08Tests: preserving unit.log when run without restart.Max Romanov1-1/+1
Introducing "unit.log.Log" class for "unit.log" file management. Moving "findall()" function into TestApplicationProto. Using "os.kill()" to send signals.
2021-04-05Tests: style.Andrei Zeliankou1-0/+1
2021-01-14Tests: added missing checks for configuration results.Andrei Zeliankou1-4/+8
2021-01-13Tests: style.Andrei Zeliankou1-1/+0
2021-01-12Tests: unit_stop() removed where possible.Andrei Zeliankou1-31/+0
Since wait_for_record() was introduced there is no need to stop Unit before parsing unit.log.
2020-12-06Tests: options moved to the separate class.Andrei Zeliankou1-1/+1
This change is necessary to separate the logic and prevent possible circular dependency.
2020-10-19Tests: fixed unit.log print.Andrei Zeliankou1-19/+21
2020-10-07Tests: minor fixes.Andrei Zeliankou1-1/+2
2020-09-16Tests: migrated to the pytest.Andrei Zeliankou1-85/+75
2020-05-15Tests: style.Andrei Zeliankou1-2/+1
2020-04-24Tests: introduced module version specification in prerequisites.Andrei Zeliankou1-1/+1
2020-03-10Tests: redirect tests output to the stdout.Andrei Zeliankou1-1/+0
2019-10-23Tests: added USR1 signal test for unit.log.Andrey Zelenkov1-37/+0
2019-10-23Tests: style.Andrey Zelenkov1-1/+3
2019-10-23Tests: check initial configuration in test_access_log.py.Andrey Zelenkov1-1/+5
2019-09-14Tests: refactored prerequisites model.Andrey Zelenkov1-1/+1
2019-08-01Tests: added delay before SIGQUIT in access_log partial tests.Andrey Zelenkov1-3/+9
This change is necessary to avoid race between client connection close and Unit close. Also "read_timeout" value decreased to speed up tests.
2019-05-30Tests: adjusted skipping tests.Andrey Zelenkov1-1/+1
2019-04-22Tests: read_timeout increased for incomplete requests.Andrey Zelenkov1-6/+8
Also fixed test without \"Host\" header.
2019-04-22Tests: using "pass" option instead of deprecated "application".Valentin Bartenev1-2/+2
2019-04-15Tests: added time to handle requests in access_log partial tests.Andrey Zelenkov1-4/+12
2019-04-09Tests: speed up tests.Andrey Zelenkov1-5/+10
2019-04-09Tests: wait_for_record() introduced.Andrey Zelenkov1-60/+31
2019-04-09Tests: simplified module checking.Andrey Zelenkov1-2/+1
2019-03-28Tests: unit module refactoring.Andrey Zelenkov1-5/+4
2019-03-26Tests: style.Andrey Zelenkov1-58/+93
2019-02-27Tests: added "Host" and "Connetion" headers where necessary.Andrey Zelenkov1-4/+4
Also minor header organizing.
2018-11-15Tests: added command line arguments parsing in tests.Andrey Zelenkov1-1/+1
Added the following command line arguments: -d, --detailed: Show detailed output for tests Usage examples: ./test/run.py --detailed python3 test/test_access_log.py --detailed python3 test/test_access_log.py -d TestUnitAccessLog.test_access_log_ipv6 -l, --log: Save unit.log after the test execution Usage examples: ./test/run.py -l python3 test/test_access_log.py -l python3 test/test_access_log.py --log TestUnitAccessLog.test_access_log_ipv6
2018-09-06Tests: used relative path for configuration.Andrey Zelenkov1-5/+5
2018-04-26Tests: response handling improved.Andrey Zelenkov1-5/+5
2018-04-23Tests: added access log test with unix socket.Andrey Zelenkov1-0/+20
2018-04-23Tests: added delay to wait for requests logging.Andrey Zelenkov1-0/+33
2018-04-18Tests: re-opening access log file.Andrey Zelenkov1-1/+36
2018-04-11Tests: access_log tests.Andrey Zelenkov1-0/+220
generated by cgit (git 2.34.1) at 2024-12-28 16:58:26 +0000