Introduction and principle analysis of Nginx

Introduction and principle analysis of Nginx

Introduction to Nginx

Nginx is an asynchronous web server that can be used as a reverse proxy, load balancing, and caching server

Basic functions:

  • Web server (common)
  • Reverse proxy server (common)
  • Mail proxy server (rarely used)
  • General TCP / UDP proxy server (four layer scheduler)


  • High performance and high concurrency
  • Modular development
  • Hot deployment, hot update
  • Low memory consumption
  • Simple configuration and maintenance

schematic diagram:
The picture is extracted from the Internet, invaded and deleted

Common commands

  • nginx startup
  • nginx -s quit stops nginx gracefully. When there is a connection, it will wait until the connection request is completed before killing the worker process
  • nginx -s reload gracefully restarts and reloads the configuration file nginx conf
  • nginx -s reopen reopens the log file, which is generally used to cut the log
  • nginx -v view version
  • nginx -t check whether the configuration file syntax of nginx is correct
  • nginx -h view help information
  • nginx -V detailed version information, including compilation parameters
  • nginx -c filename specifies the configuration file

Nginx Core Modules

  1. main core module—— Quick check of official documents
  • User (specifies the user name to run the Nginx worker process)
  • daemon (specify whether Nginx runs in the background)
  • worker_processes (defines the number of work processes, and the optimal value depends on many factors, including (but not limited to) the number of CPU cores, the number of hard drives to store data, and the loading mode)
  • error_log (define the output location and error log level of Nginx error log)
  • include (specify the location of the sub profile introduced in the main profile)
  • load_module (specify the location of the module file loaded in the main configuration file)
  • accept_mutex (on: worker processes poll and receive connections; off: notify all worker processes of new connections)
  1. Http core module—— Quick check of official documents
  • keepalive_timeout (timeout to keep active client connections)
  • Sendfile (specify whether nginx calls sendfile function (zero copy) to output files. For ordinary applications, it must be set to on. If it is used for downloading and other applications, it can be set to OFF for disk I/O heavy load applications)
  • autoindex (whether to enable directory list access, closed by default)
  1. mail core module—— Quick check of official documents

  2. stream core module—— Quick check of official documents

Nginx profile overview

  1. Main configuration file: / usr / local / nginx / conf / nginx conf
  • When nginx is started by default, the configuration file used is: installation path / conf / nginx Conf file
  • You can specify the configuration file to be read with the - c option when you start nginx
  1. nginx.conf configuration overview
  • Main: global configuration section. The main configuration section may contain the event configuration section
  • Event setting: defines the working characteristics of the event model
  • http (settings related to http protocol): defines the configuration related to http protocol
  • server (host settings)
  • Upstream (upstream server settings, mainly related configurations of reverse proxy and load balancing)
  • Location (the setting after the URL matches a specific location)

Sample Nginx configuration file

  1. Mainconfiguration section
#master: responsible for scheduling at the system management level, and worker: handling user requests
#User group of worker process
user nginx;
#Specify the number of worker processes. The default is auto: the same as the number of cpu cores
worker_processes auto;
#Define nginx error log location
error_log /var/log/nginx/error.log;
#Define the pid file path of nginx main process (the signal operations such as start, stop and restart of nginx all depend on this file)
pid /run/;
#Include sub configurations
include /usr/share/nginx/modules/*.conf;
#Used to define event driven related configurations
events {
    #Define the event driven model used by nginx
    use epoll;
    #Number of connections woker can accept
    worker_connections 1024;
    #Method for handling new connections: on: all workers are handled in turn off: all workers are notified, but only one worker is authorized to handle connections. Centros7 recommends using off 
    accept_muetx  off;
  1. http configuration section
 # http protocol level
http {           
 include mime.types;
 default_type application/octet-stream;
 keepalive_timeout 65;
 gzip on;
 #Specify whether nginx calls sendfile function (zero copy) to output files. For ordinary applications, it must be set to on. If it is used for downloading and other applications, it can be set to OFF to balance the processing speed of disk and network I/O and reduce the uptime of the system
 sendfile on;		
 #Whether to enable directory list access. It is suitable for downloading the server. It is off by default					
 autoindex off;
    #Load balancing configuration		
 	upstream {             
    #http server
    #At the server level, each server is similar to a < virtualhost in httpd. Generally speaking, it is a website >
 	server {             
	 	server_name localhost;
        #Request level, similar to < location > in httpd, is used to define the mapping relationship between URL and local file system
		location / {           
		 	root html;
		 	index index.html index.htm;

    #https server
    #Server level
 	server {             
 	    listen       443 ssl;
        server_name  localhost;

        ssl_certificate     /usr/local/nginx/ssl/nginx.crt;		
        ssl_certificate_key  /usr/local/nginx/ssl/nginx.key;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;

        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            root   html;
            index  index.html;

The https certificate generation steps are as follows:
Reference blog

  • Create a directory for storing signing certificates and enter the directory
mkdir /usr/local/nginx/ssl
cd /usr/local/nginx/ssl
  • Create server private key
openssl genrsa -des3 -out nginx.key 1024
#Enter the key and confirm the key according to the prompt, that is, nginx is generated in the current directory Key file
  • Create signing request Certificate (csr)
[root@localhost ssl]# openssl req -new -key nginx.key -out nginx.csr
Enter pass phrase for nginx.key:       ##Enter key
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [XX]:CN		##country
State or Province Name (full name) []:HB		#province
Locality Name (eg, city) [Default City]:WH			##city
Organization Name (eg, company) [Default Company Ltd]		##corporate name
Organizational Unit Name (eg, section) []	##department
Common Name (eg, your name or your server's hostname) []	 ##host name
Email Address []		##mailbox

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:		##You don't have to enter
An optional company name []:		##You don't have to enter
[root@localhost ssl]# ls
nginx.csr  nginx.key
  • Generate crt
[root@localhost ssl]# openssl x509 -req -days 365 -in nginx.csr -signkey nginx.key -out nginx.crt
Signature ok
Getting Private key
Enter pass phrase for nginx.key:
  1. mail configuration section (less used)
  • preparation
    Disable postfix service
service postfix stop
chkconfig postfix off

Modules to be configured during installation and compilation: – with mail

./configure --prefix=/date/Appsfot/nginx` --with-mail  --with-stream 
 make && make install
  • Configuration example
mail {
  #mail service name
  #Provide authentication method for POP3/IMAP user authentication
  auth_http localhost:9000/auth;
  #Enable or disable mail proxy
  proxy on;
  #Support ssl related configuration

  #Indicates that SSLv3 and TLSv1 server passwords are preferred by customers
  ssl_prefer_server_ciphers on;
  #SSL protocol used
  ssl_protocols TLSv1 SSLv3;
  #Support password (OpenSSL format)
  ssl_ciphers HIGH:!ADH:!MD5:@STRENFTH;
  #Whether all worker processes share the specified SSL cache
  ssl_session_cache shared:MAIL:10m;
  #PEM encoded SSL certificate path
  ssl_certificate /opt/;
  #PEM encoded SSL cryptographic key
  ssl_certificate_key /opt/;

  #Back end services support POP3
  pop3_capabilities TOP USER;
  #Back end services support IMAP4
  imap_capabilities IMAP4rev1 UIDPLUS QUOTA;
  #Back end services support SMTP
  smtp_capabilities PIPELINING 8BITMIME DSN;

  pop3_auth apop cram-md5;
  imap_auth login cram-md5;
  smtp_auth login cram-md5;

  server {
    listen 25;
    protocol smtp;
    timeout 120000;

  server {
    listen 465;
    protocol smtp;
    ssl on;

  server {
    listen 587;
    protocol smtp;
    starttls on;

  server {
    listen 110;
    protocol pop3;
    starttls on;

  server {
    listen 995;
    protocol pop3;
    ssl on;

  server {
    listen 143;
    protocol imap;
    starttls on;

  server {
    listen 993;
    protocol imap;
    ssl on;
  1. stream configuration section (less used)
  • preparation
    Modules to be configured during installation and compilation: – with stream
  • stream configuration example
stream {
    upstream backend {
        hash $remote_addr consistent;

        server weight=5;
        server            max_fails=3 fail_timeout=30s;
        server unix:/tmp/backend3;

    upstream dns {

    server {
        listen 12345;
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass backend;

    server {
        listen udp reuseport;
        proxy_timeout 20s;
        proxy_pass dns;

    server {
        listen [::1]:12345;
        proxy_pass unix:/tmp/stream.socket;

Note: after modifying the configuration every time, remember to check whether the format is correct and restart nginx!!!

Configure virtual host

Configuration of virtual host each server is an independent virtual host

  1. prepare documents
  • Create a sub configuration file IP. In the / etc/nginx/conf.d / directory conf, domain. conf, port. conf
  • Create presentation directory
    mkdir /data/nginx/{ip,domain,port} -pv
  • Create index. In the corresponding demo directory HTML and add differentiated content
  1. Three configuration modes
  • Based on ip address

vim /etc/nginx/conf.d/ip.conf

#Add the following
    root /data/nginx/ip/;
    index index.html;

nginx -s reload

#Test access , if the access content is correct, it is successful

  • port based
vim /etc/nginx/conf.d/port.conf
#Add the following
    listen 81;
    root /data/nginx/port/;
#After restarting nginx, test the access , that is, the content is accessed successfully
  • Based on domian (common mode in production environment)
vim /etc/nginx/conf.d/domain.conf
#Add the following
    listen 80;
    root /data/nginx/domain/;

#If the domain name is just a test domain name, you need to configure the hosts file on the test machine
#Example: append at the end of the hosts file

#Test after nginx restart visit

#Xiaokeng here: the page corresponding to the ip address will be resolved first through the domain name access, and the page corresponding to the domain name will not be resolved after the resolution. Two experiments have been done here, resulting in conflict, resulting in unexpected results. Generally, the methods of ip and domain name will not be used at the same time. The ip configuration can be successfully accessed by temporarily commenting here

Nginx tuning and its related principles (the following is a summary of beginners. Novices are welcome to give advice)

Forward proxy and reverse proxy

  1. Forward proxy

  2. Reverse proxy

Receiving process of Linux network package

Reference blog

The picture is extracted from the Internet, invaded and deleted!!

  1. Packet receiving process of network card
    As a whole, the network is the process that the high and low levels in the network cable are converted to the FIFO storage of the network card and then copied to the main memory (DDR3) of the system, which involves the network card controller

  2. Main modules involved

  • Network card controller
  • CPU
  • DMA
  • Driver
  1. Comparison of three network architectures
    The picture is extracted from the Internet, invaded and deleted!!

  2. Location in OSI model
    It belongs to physical layer and link layer

  3. The general steps of the processing flow of the data packet received by the physical network card are as follows:

  • Network card receives data packet
  • Transfer the data packet from the network card hardware cache to the server memory
  • Notification kernel processing
  • It is processed layer by layer through TCP/IP protocol
  • The application reads data from the socket buffer through read()

Linux Network IO model

  1. Blocking I/O model
  2. Non blocking I/O model
  3. I/O multiplexing model
  • select model (operation mode: traversing the underlying implementation: array bitmap time complexity: o(n))
  • poll model (operation mode: traversing the underlying implementation: linked list time complexity: o(n))
  • epoll model (operation mode: callback, underlying implementation: hash table, time complexity: o(1))
  1. Signal driven I/O model
  2. Asynchronous I/O model
  3. summary
  • Blocking IO, non blocking IO, multiplexing IO and signal driven IO are blocked when the kernel data is copied to the user space
  • The difference between blocking IO and non blocking IO is whether the originating IO request will be blocked. If yes, it is blocking IO and if not, it is non blocking io
  • The difference between synchronous IO and asynchronous IO is whether the actual IO reading and writing will block the requesting process. If yes, it is synchronous io. If it is not blocked, but the operating system helps you finish the IO operation and then returns the result to you, it is asynchronous io

TCP three handshakes and four waves

Reference blog

  1. TCP flag bit description
  • SYN - synchronous Online
  • ACK - acknowledgement confirmation
  • PSH - push transfer
  • FIN - finish end
  • RST - reset
  • URG - urgent
  • Sequence number - sequence number
  • Knowledge number - confirmation code
  1. TCP status description
  • LISTEN - LISTEN for connection requests from remote TCP ports
  • SYN-SENT - wait for matching connection request after sending connection request
  • SYN-RECEIVED - wait for confirmation of the connection request after receiving and sending a connection request
  • ESTABLISHED - represents an open connection where data can be transferred to the user
  • FIN-WAIT-1 - wait for the connection interruption request of remote TCP or the confirmation of the previous connection interruption request
  • FIN-WAIT-2 - wait for connection interruption request from remote TCP
  • CLOSE-WAIT - wait for the connection interruption request from the local user
  • CLOSING - wait for the remote TCP to confirm the connection interruption
  • LAST-ACK - wait for the confirmation of the original connection interruption request sent to the remote TCP
  • TIME-WAIT - wait enough time to ensure that the remote TCP receives an acknowledgement of the connection interruption request
  • CLOSED - no connection status
  1. Three handshakes
  • First handshake: when establishing A connection, client A sends SYN packet (SYN=j) to server B and enters SYN_SEND status, waiting for server B to confirm
  • The second handshake: when server B receives the SYN packet, it must confirm the SYN (ACK=j+1) of client A. at the same time, it also sends a SYN packet (SYN=k), that is, SYN+ACK packet. At this time, server B enters SYN_RECV status
  • The third Handshake: client A receives the SYN + ACK packet from server B and sends A confirmation packet ack (ACK=k+1) to server B. after the packet is sent, client A and server B enter the ESTABLISHED state and complete the three handshakes
  • Where, the value of confirmation number = the sending sequence number of the sender + 1 (i.e. the next sequence number expected by the receiver)
  1. Four waves
  • Client A sends A FIN to close the data transmission from client A to server B
  • Server B receives this FIN and sends back an ACK. The confirmation sequence number is the received sequence number plus 1. Like SYN, a FIN will occupy a serial number
  • Server B closes the connection with client A and sends A FIN to client A
  • Client A sends back ACK message confirmation and sets the confirmation sequence number to the received sequence number plus 1
  1. Related issues
  • The difference between shutdown() and close()
    The shutdown() function can choose to close the read channel or write channel of the full duplex connection. If the two channels are closed at the same time, the connection can no longer communicate. The close() function will close the read-write channel of the full duplex connection at the same time. In addition to closing the connection, it will also release the file descriptor occupied by the socket
  • syn queue (half connected)
  • backlog queue (full connection)

Nginx optimization (the optimization should be based on the actual application and should not be too blind)

  1. Profile optimization item
  • worker_processes
 nginx Number of processes, generally according to cpu The number of cores is determined. In the case of high concurrency, it is its multiple (eg:2 Four core cpu Total 8)
  • worker_cpu_affinity
One nginx The maximum number of file descriptors opened by the process. The theoretical value should be the maximum number of files opened( ulimit -n)And nginx Divide the number of processes, but nginx Distribution requests are not so uniform, so it's best to work with ulimit -n The values of are consistent
 Now in linux 2.6 The number of open files under the kernel is 65535, worker_rlimit_nofile 65535 should be filled in accordingly
  • worker_rlimit_nofile 00000001 00000010 00000100 00001000
Assign to each process cpu,Or assign a process to multiple processes cpu(eg: Allocate 4 processes to 4 cpu)
  • use epoll
1.Standard event model
select,poll It belongs to the standard event model. If there is no more effective method in the current system, nginx Will choose select or poll
2.Efficient event model 
squeue: Apply to FreeBSD 4.1+, OpenBSD 2.9+, NetBSD 2.0 and MacOS X, Dual processor MacOS X System use kqueue May cause kernel crash
epoll: Apply to Linux Kernel 2.6 System version and later

  • worker_connections 65535
each worker Maximum number of connections allowed by the process
 Theoretically, each nginx The maximum number of connections to the server is worker_processes*worker_connections
  • keepalive_timeout 60
keepalive Timeout
  • client_header_buffer_size 4k
 The buffer size of the client request header can be set according to the paging size of your system. Generally, the size of a request header will not exceed 1 k,However, because the general system paging is greater than 1 k,So set the page size here.

The paging size can be determined by using the command getconf PAGESIZE see
  • open_file_cache max=65535 inactive=60s
This will specify the cache for the open file, which is not enabled by default, max Specify the number of caches. It is recommended that the number be consistent with the number of open files, inactive It refers to how long it takes to delete the cache after the file has not been requested
  • open_file_cache_valid 80s
Specifies how often to check the cache for valid information
  • open_file_cache_min_uses 1
appoint open_file_cache In the instruction inactive The minimum number of times the file is used within the parameter time. If the limit is exceeded, the file descriptor is always opened in the cache( eg: If there is a file in inactive (if it is not used once in time, it will be removed)
  1. Optimization of system kernel parameters
net.ipv4.tcp_max_tw_buckets = 6000
timewait The default is 180000

net.ipv4.ip_local_port_range = 1024 65000
 Port range allowed to be opened by the system

net.ipv4.tcp_tw_recycle = 1
 Enable timewait Rapid recovery

net.ipv4.tcp_tw_reuse = 1
 Turn on reuse. Allow will TIME-WAIT sockets Reuse for new TCP connect
  1. Anti theft chain
  • In nginx Add a location configuration item to the conf file
  • Configuration example
#Regular matching requires files with anti-theft chains, or set directories with anti-theft chains
location ~* ^.+\.(gif|jpg|png|swf|flv|rar|zip)$ { 
       #Configure directly accessible services
        valid_referers none blocked;
       #Those not in the above configuration need to jump the anti-theft chain 
        if ($invalid_referer) {
            return 404;
            #rewrite ^/ ; (a friendly prompt picture can be configured. Note: the picture is placed on the Internet)
        access_log off;
        root html/www;
        expires 1d;

Installation and configuration example

Please refer to my blog

Tags: Linux network Nginx

Posted by dough boy on Wed, 11 May 2022 22:59:56 +0300