php+nginx服务发生500、502错误如何排查
php+nginx服务发生500、502错误如何排查
这篇文章主要介绍了php+nginx服务发生500、502错误如何排查的相关知识,内容详细易懂,操作简单快捷,具有一定借鉴价值,相信大家阅读完这篇php+nginx服务发生500、502错误如何排查文章都会有所收获,下面我们一起来看看吧。
概述
当线上的服务中访问中出现500或者502错误时,需要紧急处理,排查问题,该怎么做?可以通过分析一些错误日志或者跟踪php-fpm进程来进行问题定位。
nginx error_log
nginx的error_log在nginx的配置文件中定义的
server { listen 80; server_name localhost; root /var/www; access_log /users/jiao/logs/default.access.log; error_log /users/jiao/logs/default.error.log; location / { index index.html index.htm index.php; autoindex on; } location = /info { allow 127.0.0.1; deny all; rewrite (.*) /.info.php; } location ~ .php$ { root /var/www; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param script_filename /var/www$fastcgi_script_name; include /usr/local/etc/nginx/fastcgi_params; } }
查看error_log
➜ tail /users/jiao/logs/default.error.log
2019/07/17 11:08:18 [error] 77416#0: *76 kevent() reported about an closed connection (54: connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "get / http/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "localhost"
发现出现了connection reset by peer,连接被重置了,此时可以再查看php-fpm的error_log进一步分析问题
php-fpm error_log
php-fpm的error_log在php-fpm.conf文件中配置中定义的
; error log file ; if it's set to "syslog", log is sent to syslogd instead of being written ; in a local file. ; note: the default prefix is /usr/local/var ; default value: log/php-fpm.log error_log = log/php-fpm.log
error_log里面的内容是这样的
➜ tail /usr/local/var/log/php-fpm.log [17-jul-2019 10:49:54] notice: [pool www] child 81948 started [17-jul-2019 11:08:18] warning: [pool www] child 77537, script '/var/www/index.php' (request: "get /index.php") execution timed out (3.801267 sec), terminating [17-jul-2019 11:08:18] warning: [pool www] child 77537 exited on signal 15 (sigterm) after 1503.113967 seconds from start [17-jul-2019 11:08:18] notice: [pool www] child 94339 started
可以看到是请求/var/www/index.php文件出现了超时
dtruss
dtruss是动态跟踪命令,可以根据pid,name跟踪进程
mac环境下使用dtruss,linux环境可以使用strace,pstack
➜ dtruss usage: dtruss [-acdefholls] [-t syscall] { -p pid | -n name | command | -w name }
-p pid # examine this pid -n name # examine this process name -t syscall # examine this syscall only -w name # wait for a process matching this name -a # print all details -c # print syscall counts -d # print relative times (us) -e # print elapsed times (us) -f # follow children -l # force printing pid/lwpid -o # print on cpu times -s # print stack backtraces -l # don't print pid/lwpid -b bufsize # dynamic variable buf size
eg,
dtruss df -h # run and examine "df -h" dtruss -p 1871 # examine pid 1871 dtruss -n tar # examine all processes called "tar" dtruss -f test.sh # run test.sh and follow children
跟踪php-fpm:sudo dtruss -a -n php-fpm
此时访问web页面,就可以看到跟踪内容
21416/0x3479b6: 1559 63 3 getrusage(0x0, 0x7ffee1ec0760, 0x0) = 0 0 21416/0x3479b6: 1561 4 0 getrusage(0xffffffffffffffff, 0x7ffee1ec0760, 0x0) = 0 0 21416/0x3479b6: 1627 77 17 poll(0x7ffee1ec08c0, 0x1, 0x1388) = 1 0 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 21416/0x3479b6: 1872 29 24 lstat64("/var/www/index.php ", 0x7ffee1ecff38, 0x0) = 0 0 21416/0x3479b6: 1884 9 6 lstat64("/var/www ", 0x7ffee1ecfdf8, 0x0) = 0 0 21416/0x3479b6: 1889 6 3 lstat64("/var ", 0x7ffee1ecfcb8, 0x0) = 0 0 21416/0x3479b6: 1899 12 8 readlink("/var ", 0x7ffee1ed0090, 0x400) = 11 0 21416/0x3479b6: 1905 6 4 lstat64("/private/var ", 0x7ffee1ecfb78, 0x0) = 0 0 21416/0x3479b6: 1917 6 3 lstat64("/private ", 0x7ffee1ecfa38, 0x0) = 0 0 21416/0x3479b6: 2178 18 14 stat64("/var/www/.user.ini ", 0x7ffee1ed0240, 0x0) = -1 err#2 21416/0x3479b6: 2217 5 1 setitimer(0x2, 0x7ffee1ed07e0, 0x0) = 0 0 21416/0x3479b6: 2225 4 0 sigaction(0x1b, 0x7ffee1ed0788, 0x7ffee1ed07b0) = 0 0 21416/0x3479b6: 2237 5 1 sigprocmask(0x2, 0x7ffee1ed0804, 0x0) = 0x0 0 21416/0x3479b6: 3643 48 40 open_nocancel(". ", 0x0, 0x1) = 5 0 21416/0x3479b6: 3648 7 3 fstat64(0x5, 0x7ffee1ed0110, 0x0) = 0 0 21416/0x3479b6: 3653 7 2 fcntl_nocancel(0x5, 0x32, 0x10f252158) = 0 0 21416/0x3479b6: 3661 12 7 close_nocancel(0x5) = 0 0 21416/0x3479b6: 3670 10 7 stat64("/usr/local/var ", 0x7ffee1ed0080, 0x0) = 0 0 21416/0x3479b6: 3681 11 8 chdir("/var/www ", 0x0, 0x0) = 0 0 21416/0x3479b6: 3698 4 0 setitimer(0x2, 0x7ffee1ed02d0, 0x0) = 0 0 21416/0x3479b6: 3710 6 3 fcntl(0x3, 0x8, 0x10f3fd858) = 0 0 21416/0x3479b6: 3733 9 6 stat64("/private/var/www/index.php ", 0x7ffee1ecff10, 0x0) = 0 0 74904/0x332630: 723125 1073381 19 kevent(0x9, 0x0, 0x0) = 0 0 74902/0x332629: 770666 1073387 17 kevent(0x8, 0x0, 0x0) = 0 0 74904/0x332630: 723165 1061954 20 kevent(0x9, 0x0, 0x0) = 0 0 74902/0x332629: 770709 1061954 20 kevent(0x8, 0x0, 0x0) = 0 0 74904/0x332630: 723201 1074786 16 kevent(0x9, 0x0, 0x0) = 0 0 74902/0x332629: 770747 1074783 16 kevent(0x8, 0x0, 0x0) = 0 0 74904/0x332630: 723229 1069141 13 kevent(0x9, 0x0, 0x0) = 0 0 74902/0x332629: 770777 1069145 11 kevent(0x8, 0x0, 0x0) = 0 0 21416/0x3479b6: 3942 3902233 7 __semwait_signal(0x703, 0x0, 0x1) = -1 err#4 74902/0x332629: 770814 103 25 kill(21416, 15) = 0 0 dtrace: error on enabled probe id 2172 (id 161: syscall::write:return): invalid kernel access in action #13 at dif offset 68 dtrace: error on enabled probe id 2172 (id 161: syscall::write:return): invalid kernel access in action #13 at dif offset 68 74902/0x332629: 771325 7 2 sigreturn(0x7ffee1ecfc40, 0x1e, 0xc1a4b78e0404663a) = 0 err#-2 74902/0x332629: 771336 7 3 kevent(0x8, 0x0, 0x0) = 1 0 dtrace: error on enabled probe id 2174 (id 159: syscall::read:return): invalid kernel access in action #13 at dif offset 68 74902/0x332629: 771352 11 7 wait4(0xffffffffffffffff, 0x7ffee1ed0748, 0x3) = 21416 0 dtrace: error on enabled probe id 2172 (id 161: syscall::write:return): invalid kernel access in action #13 at dif offset 68 74902/0x332629: 773511 1957 1899 fork() = 28060 0 28060/0x3754c5: 125: 0: 0 fork() = 0 0 28060/0x3754c5: 128 9 2 bsdthread_register(0x7fff6774c418, 0x7fff6774c408, 0x2000) = -1 err#22 dtrace: error on enabled probe id 2172 (id 161: syscall::write:return): invalid kernel access in action #13 at dif offset 68 74902/0x332629: 773737 4 1 wait4(0xffffffffffffffff, 0x7ffee1ed0748, 0x3) = 0 0 74902/0x332629: 773742 6 3 read(0x5, "