Maggie's blog

nginx related knowledge

system

CENTOS>=7.0, number of digits X64 CENTOS 7.2

nginx advantages

IO multiplexing: IO operations of multiple descriptors can be completed concurrently and alternately in one thread, reusing threads.
After Nginx is started, there will be a master process and multiple independent worker processes.
Receive signals from the outside world and send signals to each worker process. Each process may handle this connection.
The master process can monitor the running status of the worker process. When the worker process exits (under abnormal circumstances), a new worker process will be automatically started.
The number of worker processes is usually set to the number of machine CPU cores.Because more workers will only cause processes to compete with each other for the CPU, resulting in unnecessary context switching.
Using multi-process mode not only improves the concurrency rate, but also makes the processes independent of each other. If one worker process hangs up, it will not affect other worker processes.
CPU affinity binds the CPU core and the nginx worker process together, allowing each worker process to be executed on one CPU, thereby reducing CPU switching and improving cache hit rate and performance.
sendfile zero-copy transmission mode
Under the default configuration, after Nginx reads the local file, it will first read the hard disk file from the hard disk to the Nginx file buffer during network transmission. The operation process is: hard disk → kernel file buffer → application buffer.Then write the data in the Nginx file buffer to the network interface. The operation process is: application buffer → kernel network buffer → network interface.
During the network transmission process, Nginx's local file went through the above two operations of reading and writing. Both operations stored the same data in the kernel buffer.In order to improve the efficiency of file transmission, the kernel provides zero-copy technology, which supports files to directly exchange open file handles in the kernel buffer without repeatedly copying the file content to the buffer. The process of the above two operations becomes: hard disk → kernel file buffer → kernel network buffer → network interface.
Zero-copy technology reduces the number of file reads and writes and increases the network transmission speed of local files.The default size of the kernel buffer is 4096B.

nginx version

Mainline version development version
Stable version stable version
Legacy versions historical versions

nginx download address nginx linux package

YUM installation under CentOS
Path:/etc/yum.repos.d/nginx.repo
File content:

bash

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1

Command:

bash

yum install nginx -y
nginx -v
nginx -V

Configuration files and uses

View configuration files and directories

bash

rpm -ql nginx

Configuration files and uses

type	path	purpose
Configuration file	/etc/logrotate.d/nginx	Log cutting for logrotate service
Configuration file	/etc/nginx /etc/nginx/nginx.conf /etc/nginx/conf.d /etc/nginx/conf.d/default.conf	Main configuration file
Configuration file	/etc/nginx/fastcgi_params /etc/nginx/scgi_params /etc/nginx/uwsgi_params	cgi configuration, fastcgi configuration
Configuration file	/etc/nginx/koi-utf /etc/nginx/koi-win /etc/nginx/win-utf	Encoding conversion mapping conversion file
Configuration file	/etc/nginx/mime.types	Set the corresponding relationship between the Content-Type and extension of the http protocol
Configuration file	/usr/lib/systemd/system/nginx-debug.service /usr/lib/systemd/system/nginx.service /etc/sysconfig/nginx /etc/sysconfig/nginx-debug	Used to configure the system daemon manager management mode
Configuration file	/etc/nginx/modules /usr/lib64/nginx/modules	nginx module directory
Command	/usr/share/doc/nginx-1.14.0 /usr/share/doc/nginx-1.14.0/COPYRIGHT	nginx manual and help files
Directory	/var/cache/nginx	nginx cache directory
Directory	/var/log/nginx	nginx log directory

Installation directory and path

bash

--prefix=/etc/nginx # 安装目录
--sbin-path=/usr/sbin/nginx # 可执行文件
--modules-path=/usr/lib64/nginx/modules # 安装模块
--conf-path=/etc/nginx/nginx.conf  # 配置文件路径
--error-log-path=/var/log/nginx/error.log  # 错误日志
--http-log-path=/var/log/nginx/access.log  # 访问日志
--pid-path=/var/run/nginx.pid # 进程ID
--lock-path=/var/run/nginx.lock # 加锁对象

Temporary files retained by nginx when executing the corresponding module

bash

--http-client-body-temp-path=/var/cache/nginx/client_temp # 客户端请求体临时路径
--http-proxy-temp-path=/var/cache/nginx/proxy_temp # 代理临时路径
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp 
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp 
--http-scgi-temp-path=/var/cache/nginx/scgi_temp

Set the user and user group for nginx process startup

bash

--user=nginx   # 指定用户
--group=nginx  # 指定用户组

Setting additional parameters will be added to the CFLAGS variable. The CFLAGS variable is used to store optimization parameters during C language compilation.

bash

--with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong

Set link file parameters and link system libraries.
Defines additional options to be passed to the C linker command line.
PCRE library, needs to be specified –with-ld-opt="-L /usr/local/lib"

bash

--with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

Other parameters

bash

--with-compat 
--with-file-aio 
--with-threads 
--with-http_addition_module 
--with-http_auth_request_module 
--with-http_dav_module 
--with-http_flv_module 
--with-http_gunzip_module 
--with-http_gzip_static_module 
--with-http_mp4_module 
--with-http_random_index_module 
--with-http_realip_module 
--with-http_secure_link_module 
--with-http_slice_module 
--with-http_ssl_module 
--with-http_stub_status_module 
--with-http_sub_module 
--with-http_v2_module 
--with-mail 
--with-mail_ssl_module 
--with-stream 
--with-stream_realip_module 
--with-stream_ssl_module 
--with-stream_ssl_preread_module 
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC'

Main configuration file

bash

/etc/nginx/nginx.conf # 核心配置文件
/etc/nginx/conf.d/*.conf # 包含conf.d目录下面的所有配置文件
/etc/nginx/conf.d/default.conf # 默认http服务器配置文件

cgi configuration file

CGI is common gateway interface (common gateway interface)
Web Server can pass dynamic requests to applications such as php, jsp, python and perl through the cgi protocol
FastCGI is actually CGI with some extended functions. It is an improvement of CGI and describes a standard for transmitting data between client and Web server programs.
The SCGI protocol is a replacement for the CGI (Common Gateway Interface) protocol. It is an interface standard for applications and HTTP servers, similar to FastCGI, but it is designed to be easier to implement.
uwsgi is a web server that implements the WSGI protocol, uwsgi, http and other protocols

Path	Purpose
/etc/nginx/fastcgi_params	fastcgi configuration
/etc/nginx/scgi_params	scgi configuration
/etc/nginx/uwsgi_params	uwsgi configuration

Encoding conversion mapping conversion file

These three files are encoding conversion mapping files, used to convert one encoding to another when outputting content to the client.
koi8-r is a Cyrillic 8-bit encoding for Russian and Bulgarian.Before Unicode became popular, KOI8-R was the most widely used Russian encoding, and its usage rate was even higher than ISO/IEC 8859-5.These three documents exist because the authors are Russians.

Path	Purpose
/etc/nginx/koi-utf	koi8-r <--> utf-8
/etc/nginx/koi-win	koi8-r <--> windows-1251
/etc/nginx/win-utf	windows-1251 < -- > utf-8

extension file

/etc/nginx/mime.types sets the corresponding relationship between Content-Type and extension of http protocol

Daemon management

Used to configure the system daemon manager management method

Path
/usr/lib/systemd/system/nginx-debug.service
/usr/lib/systemd/system/nginx.service
/etc/sysconfig/nginx
/etc/sysconfig/nginx-debug

bash

systemctl restart nginx.service

nginx module directory

/etc/nginx/modules is the most basic shared library and kernel module.
The purpose is to store shared libraries of binary files such as /bin and /sbin used to start the system and execute commands of the root file system, or store 32-bit or 64-bit (file command to view) /usr/lib64/nginx/modules 64-bit shared library

nginx manual and help files

path	purpose
/usr/share/doc/nginx-1.14.2	Help Documentation
/usr/share/doc/nginx-1.14.0/COPYRIGHT	Copyright Statement
/usr/share/man/man8/nginx.8.gz	Manual

nginx service startup management executable file

Path	Purpose
/usr/sbin/nginx	Executable command
/usr/sbin/nginx-debug	Debug and execute executable commands

nginx content configuration

nginx configuration syntax

bash

# 使用#可以添加注释,使用$符号可以使用变量
# 配置文件由指令与指令块组成,指令块以{}将多条指令组织在一起
http {
# include语句允许把多个配置文件组合起来以提升可维护性    
    include       mime.types;
# 每条指令以;(分号)结尾，指令与参数之间以空格分隔
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
# 有些指令可以支持正则表达式        
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

Global and service configuration

Classification	Configuration items	Function
Global	user	Set the system user for nginx service
Global	worker_processes	Number of worker processes, generally the same as the number of CPUs
Global	error_log	nginx error log
global	pid	pid when nginx service starts

event configuration

Classification	Configuration items	Function
events	worker_connections	Maximum number of connections allowed per process 10000
events	use	Specify which model to use (select/poll/epoll). It is recommended to let nginx automatically select. Linux kernel 2.6 and above can generally use epoll to improve performance.

http configuration

The configuration file /etc/nginx/nginx.conf, multiple servers can be configured under one HTTP.

bash

user  nginx;   # 设置nginx服务的系统使用用户  
worker_processes  1;  # 工作进程数,一般和CPU数量相同 

error_log  /var/log/nginx/error.log warn;   # nginx的错误日志  
pid        /var/run/nginx.pid;   # nginx服务启动时的pid

events {
    worker_connections  1024; # 每个进程允许的最大连接数 10000
}

http {
    include       /etc/nginx/mime.types; # 文件后缀和类型类型的对应关系
    default_type  application/octet-stream; # 默认content-type

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';  # 日志记录格式

    access_log  /var/log/nginx/access.log  main; # 默认访问日志

    sendfile        on; # 启用sendfile
    #tcp_nopush     on; # 懒发送

    keepalive_timeout  65; # 超时时间是65秒

    #gzip  on;gzip压缩

    include /etc/nginx/conf.d/*.conf; # 包含的子配置文件
}

server configuration

The file is /etc/nginx/conf.d/default.conf, and multiple locations can be configured under one server.

bash

server {
    listen       80;  # 监听的端口号
    server_name  localhost;  # 用域名方式访问的地址

    #charset koi8-r; # 编码
    #access_log  /var/log/nginx/host.access.log  main;  # 访问日志文件和名称

    location / {
        root   /usr/share/nginx/html;  # 静态文件根目录
        index  index.html index.htm;  # 首页的索引文件
    }

    #error_page  404              /404.html;  # 指定错误页面

    # redirect server error pages to the static page /50x.html
    # 把后台错误重定向到静态的50x.html页面
    error_page   500 502 503 504  /50x.html; 
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    # 代理PHP脚本到80端口上的apache服务器
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    # 把PHP脚本9000端口上监听的FastCGI服务
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    # 不允许访问.htaccess文件
    # location ~ /\.ht {
    #     deny  all;
    # }
}

Systemd configuration

The system startup and server daemon process manager is responsible for activating system resources, server processes and other processes when the system is started or running. According to management, the letter d is the abbreviation of daemon.

Configuration directory

Configuration Directory	Purpose
/usr/lib/systemd/system	The main startup script settings for each service, similar to the previous /etc/initd.d
/run/system/system	Service scripts generated during system execution will be run before the directories above
/etc/system/system	The execution script created by the administrator, similar to the function of /etc/rc.d/rcN.d/Sxx, runs first than the above directory. Among the three, this directory has the highest priority

systemctl

Configure nginx systemctl command

The main command for monitoring and controlling systemd is systemctl
This command can be used to view system status and manage systems and services

bash

命令：systemctl  command name.service
启动：service name start –> systemctl start name.service
停止：service name stop –> systemctl stop name.service
重启：service name restart –> systemctl restart name.service
状态：service name status –> systemctl status name.service

Start and reload

bash

systemctl restart nginx.service
systemctl reload nginx.service
nginx -s reload

Log related

/var/log/nginx/access.log access log /var/log/nginx/error.log error log

log_format

Type	Usage
Syntax	log_format name [escape=default[json] string]
Default	log_format combined ...
Context	http

log_format example
ngx_http_log_module built-in variables

Name	Meaning
$remote_addr	Client address
$remote_user	Client user name
$time_local	Access time and time zone
$request	request line
$status	HTTP request status
$body_bytes_sent The size of the file content sent to the client

HTTP request variables

Pay attention to convert - into an underscore, for example, User-Agent corresponds to $http_user_agent

Name	Meaning	Example
arg_PARAMETER	Request parameters	$arg_name
http_HEADER	Request header	$http_referer $http_host $http_user_agent $http_x_forwarded_for(agent process)
sent_http_HEADER	Response header	sent_http_cookie

IP1->IP2 (proxy)->IP3 will record the proxy process of the IP address

http_x_forwarded_for=Client IP,Proxy(1) IP,Proxy(2) IP

Define log format

bash

 # 定义一种日志格式 
 log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

 log_format  zfpx  '$arg_name $http_referer sent_http_date"';
 # 指定写入的文件名和日志格式
 access_log  /var/log/nginx/access.log  main;

View log

bash

tail -f /var/log/nginx/access.log

221.216.143.110 - - [09/Jun/2018:22:41:18 +0800] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36" "-"

core module

Monitor the status of ngin client Module name: --with-http_stub_status_module

bash

Syntax: stub_status on/off;
Default: -
Context: server->location

Actual combat
File: /etc/nginx/conf.d/default.conf

bash

server {
+    location /status{
+       stub_status  on;
+    }

bash

systemctl reload nginx.service

http://192.171.207.104/status

Active connections: 2            
server accepts handled requests  
 3 3 10 
Reading: 0 Writing: 1 Waiting: 1

Parameter	Meaning
Active	connections
accepts	Total number of connections processed
handled	Number of successfully created handshakes
requests	Total number of requests processed
Reading	Number of Header information read to the client
Writing	The number of header information returned to the client
Waiting	When keep-alive is turned on, this value is equal to active – (reading + writing)

Randomly select a home page to display in the root directory

Module name: --with-http_random_index_module

bash

Syntax: random_index on/off;
Default: off
Context: location

Actual combat
/etc/nginx/conf.d/default.conf

bash

+    location / {
+       root /opt/app;
+       random_index on;
+    }

content replacement
Module name: --with-http_sub_module

bash

文本替换
Syntax: sub_filter string replacement;
Default: --
Context: http,service,location

只匹配一次
Syntax: sub_filter_once on|off;
Default: off
Context: http,service,location

Actual combat
File location: /etc/nginx/conf.d/default.conf

bash

location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
+        sub_filter 'world' 'mamingjuan';
+        sub_filter_once off;
    }

Request limits
module name
--with-limit_conn_module connection frequency limit
--with-limit_req_module request frequency limit
A TCP request generates at least one HTTP request
SYN > SYN,ACK->ACK->REQUEST->RESPONSE->FIN->ACK->FIN->ACK

Apache's ab command simulates multi-threaded concurrent requests to test server load pressure. It can also test the pressure of other web servers such as nginx, lighthttp, IIS, etc. Among them, -n is the total number of requests and -c is the number of concurrent requests.

bash

ab -n 40 -c 20 http://127.0.0.1/

Request limits
Module: limit_req_zone

bash

# 可以以IP为key zone为空间的名称 size为申请空间的大小
Syntax: limit_req_zone key zone=name:size rate=rate;   
Default: --
Context: http(定义在server以外)

Module: limit_req

bash

# zone名称 number限制的数量
Syntax: limit_req  zone=name [burst=number] [nodelay];
Default: --
Context: http,server,location

Case

bash

limit_req_zone $binary_remote_addr zone=req_zone:1m rate=1r/s;
server {
  location /{
      limit_req req_zone;
      # 缓存区队列burst=3个,不延期，即每秒最多可处理rate+burst个.同时处理rate个
      limit_req zone=req_zone burst=3 nodelay;
  }
}

$binary_remote_addr represents the remote IP address
zone=req_zone:10m means that the size of a memory area is 10m, and the name is set to req_zone
rate=1r/s means the request rate is 1 request per second

zone=req_zone means that the global setting corresponding to this parameter is the memory area of req_zone.

burst=3 indicates the length of the request queue
nodelay means no delay

Connection limits

Module: limit_conn_zone

bash

# 可以以IP为key zone为空间的名称 size为申请空间的大小
Syntax: limit_conn_zone key zone=name:size;   
Default: --
Context: http(定义在server以外)

Module: limit_conn

bash

# zone名称 number限制的数量
Syntax: limit_conn  zone number;
Default: --
Context: http,server,location

Case

bash

limit_conn_zone $binanry_remote_addr zone=conn_zone:1m;
server {
  location /{
      limit_conn conn_zone 1;
  }
}

Indicates that when using IP as the key to restrict each IP from accessing files, at most one can be online, otherwise the rest will be returned as unavailable.

access control
IP-based access control -http_access_module
User-based trust login -http_auth_basic_module

Module: http_access_module

bash

Syntax: allow address|all;
Default: --
Context: http,server,location,limit_except


Syntax: deny address|CIDR|all;
Default: --
Context: http,server,location,limit_except

bash

server {
+ location ~ ^/admin.html{
+      deny 192.171.207.100;
+      allow all;
+    }
}    

server {
+ location ~ ^/admin.html{
+    if ($http_x_forwarded_for !~* "^8\.8\.8\.8") {
+       return 403;
+    }
}

Symbol	Meaning
=	Strict match.If this query matches, the search is stopped and the request is processed immediately.
~	is case-sensitive matching (regular expressions available)
!~	Mismatch for case sensitivity
~*	Case-insensitive matching (regular expressions available)
!~*	is a case-insensitive mismatch
^~	If used with a regular string, this tells nginx not to test the regular expression if the path matches.

Module: http_auth_basic_module

bash

Syntax: auth_basic string|off;
Default: auth_basic off;
Context: http,server,location,limit_except

Syntax: auth_basic_user_file file;
Default: -;
Context: http,server,location,limit_except

bash

htpasswd -c /etc/nginx/users.conf zhangsan

bash

server {
+    auth_basic '请登录';
+    auth_basic_user_file /etc/nginx/users.conf;

Static resource web service

Static resources: Generally, the client sends a request to the web server. The web server retrieves the corresponding file from the memory and returns it to the client. The client parses and renders it for display.
Dynamic resources: Generally, for dynamic resources requested by the client, the request is first handed over to the web container, and the web container connects to the database. After the database processes the data, the content is handed over to the web server, and the web server returns it to the client for parsing and rendering processing.

Type	Type
Browser rendering	HTML, CSS, JS
Pictures	JPEG, GIF, PNG
Video	FLV, MPEG
Download file	Word, Excel

CDN

The full name of CDN is Content Delivery Network, which is content distribution network.
The CDN system can redirect the user's request to the service node closest to the user in real time based on comprehensive information such as network traffic and the connection and load status of each node, as well as the distance to the user and response time.Its purpose is to enable users to obtain the required content nearby, solve the congestion situation of the Internet network, and improve the response speed of users' access to the website.

Configuration syntax

sendfile sends files without going through the user kernel

Type	Type
Syntax	sendfile on / off
Default	sendfile off;
Context	http,server,location,if in location

tcp_nopush improves the transmission efficiency of network packets when sendfile is turned on.

Type	Type
Syntax	tcp_nopush on / off
Default	tcp_nopush off;
context	http,server,location

tcp_nodelay improves the real-time performance of network packet transmission under keepalive connections.

Type	Type
Syntax	tcp_nodelay on / off
Default	tcp_nodelay on;
context	http,server,location

Gzip compressed files can save bandwidth and improve network transmission efficiency

Type	Type
Syntax	gzip on / off
Default	gzip off;
context	http,server,location

gzip_comp_level The higher the compression ratio, the smaller the file is compressed.

Type	Type
Syntax	gzip_comp_level level
Default	gzip_comp_level 1;
context	http,server,location

gzip_http_version compressed HTTP version

Type	Type
Syntax	gzip_http_version 1.0/1.1
Default	gzip_http_version 1.1;
context	http,server,location

http_gzip-static_module first searches the disk to see if the .gz file with the same name exists, saving CPU compression time and performance loss.

Type	Type
Syntax	gzip_static on/off
Default	gzip_static off;
context	http,server,location

bash

    location ~ .*\.(jpg|png|gif)$ {
        gzip off;# 关闭压缩
        root /data/www/images;
    }

    location ~ .*\.(html|js|css)$ {
        gzip on; # 启用压缩
        gzip_min_length 1k;    # 只压缩超过1K的文件
        gzip_http_version 1.1; # 启用gzip压缩所需的HTTP最低版本
        gzip_comp_level 9;     # 压缩级别，压缩比率越高，文件被压缩的体积越小
        gzip_types  text/css application/javascript;# 进行压缩的文件类型
        root /data/www/html;
    }

    location ~ ^/download {
        gzip_static on; # 启用压缩
        tcp_nopush on;  # 不要着急发，攒一波再发
        root /data/www; # 注意此处目录是`/data/www`而不是`/data/www/download`
    }

browser cache

Verify whether the local cache has expired

Type	Type
Check whether it has expired	Expires, Cache-Control(max-age)
Etag	Etag
Last-Modified	Last-Modified

expires
Add Cache-Control and Expires headers

Type	Type
Syntax	expires time
Default	expires off;

Context http,server,location

bash

location ~ .*\.(jpg|png|gif)$ {
        expires 24h;
}

Cross domain

Type	Type
Syntax	add_header name value
Default	add_header --;
context	http,server,location

bash

location ~ .*\.json$ {
        add_header Access-Control-Allow-Origin http://localhost:3000;
        add_header Access-Control-Allow-Methods GET,POST,PUT,DELETE,OPTIONS;
        root /data/json;
    }

javascript

let xhr = new XMLHttpRequest();
xhr.open('GET', 'http://47.104.184.134/users.json', true);
xhr.onreadystatechange = function () {
    if (xhr.readyState == 4 && xhr.status == 200) {
        console.log(xhr.responseText);
    }
}
xhr.send();

Anti-hotlinking

Prevent website resources from being misappropriated, ensure information security, prevent excessive traffic, and distinguish which requests are abnormal user requests. Use http_refer to prevent hotlinking

Type	Type
Syntax	valid_referers none block server_names string...
Default	-
Context	server,location

bash

location ~ .*\.(jpg|png|gif)$ {
        expires 1h;
        gzip off;
        gzip_http_version 1.1;
        gzip_comp_level 3;
        gzip_types image/jpeg image/png image/gif;
        # none没有refer blocked非正式HTTP请求 特定IP
+       valid_referers none blocked 47.104.184.134;
+       if ($invalid_referer) { # 验证通过为0，不通过为1
+           return 403;
+       }
        root /data/images;
    }

bash

-e, --referer       Referer URL (H)
curl -e "http://www.baidu.com" http://192.171.207.104/girl.jpg
curl -e "192.171.207.100" http://192.171.207.104/girl.jpg

agency service

The forward proxy is the client proxy, which acts as a proxy for the client. The server does not know the client that actually initiated the request.
A reverse proxy is a server-side proxy that acts as a proxy for the server. The client does not know the server that actually provides services.

Configuration

Type	Type
Syntax	proxy_pass URL
Default	-
context	server,location

reverse proxy
The actual operation mode of Reverse Proxy is to use a proxy server to accept connection requests on the Internet, then forward the request to a server on the internal network, and return the results obtained from the server to the client requesting a connection on the Internet. At this time, the proxy server appears as a server to the outside world.
The role of reverse proxy:
(1) Ensure the security of the intranet and prevent web attacks. Large websites usually use reverse proxy as the public network access address, and the web server is the intranet.

(2) Load balancing, optimizing the load of the website through reverse proxy server

bash

resolver 8.8.8.8;# 谷歌的域名解析地址
location ~ ^/api {
    # $http_host 要访问的主机名 $request_uri请求路径
    proxy_pass http://127.0.0.1:3000;
}

Press the Win+R system hotkey to open the run window, enter the ipconfig /flushdns command and press Enter to clear the computer's DNS cache.

forward proxy

The forward proxy is like a springboard, the proxy accesses external resources.
Purpose of forward proxy:
(1) Access resources that were originally inaccessible, such as google
(2) Can be cached to speed up access to resources
(3) Authorize client access and authenticate online
(4) The agent can record user access records (online behavior management) and hide user information from the outside.

bash

location / {
        proxy_pass http://$http_host$request_uri;
    }

load balancing

The use of clusters is a common means for websites to solve high concurrency and massive data problems.
When a server's processing power and storage space are insufficient, don't try to change to a more powerful server. For large websites, no matter how powerful the server is, it cannot meet the website's continued growth in business needs.
In this case, a more appropriate approach is to add a server to share the access and storage pressure of the original server.Through the load balancing scheduling server, access requests from the browser are distributed to any server in the application server cluster. If there are more users, more application servers will be added to the cluster so that the load pressure of the application server no longer becomes the bottleneck of the entire website.

upstream

nginx forwards the request to a set of upstream service pools in the background
Nginx can be configured to proxy multiple servers. When one server goes down, the system can still be available.

Polling(weight)

Specify the polling probability, and the weight is proportional to the access ratio, which is used when the performance of the back-end server is uneven.By default, when weight is not specified, the weight of each server is the same, and each request is assigned to a different back-end server one by one in chronological order. If the back-end server goes down, it can be automatically eliminated.

bash

upstream bakend {
    server 192.168.1.10 weight=1;
    server 192.168.1.11 weight=2;
}

ip_hash

Type	Type
Syntax	upstream name {}
Default	-
context	http

bash

upstream mamingjuan {
  ip_hash;
  server localhost:3000;
  server localhost:4000;
  server localhost:5000;
}

server {
    location / {
        proxy_pass http://mamingjuan;
    }
}

Backend server debugging status

Status	Description
down	does not participate in load balancing
backup	Backup server
max_fails	Number of allowed request failures
fail_timeout	The time the service is suspended after max_fails failure
max_conns	Limit the maximum number of connections received

bash

upstream mamingjuan {
  server localhost:3000 down;
  server localhost:4000 backup;
  server localhost:5000 max_fails=1 fail_timeout=10s;
}

Distribution method

Type	Type
Polling (default)	Each request is assigned to a different backend server one by one in chronological order. If the backend server goes down, it can be automatically eliminated.
weight (weighted polling)	Specifies the polling probability, weight is proportional to the access ratio, and is used when the back-end server performance is uneven.
ip_hash	Each request is allocated according to the hash result of the access IP, so that each visitor has fixed access to a back-end server, which can solve the session problem.
url_hash (third party)	Distribute requests according to the URL address visited, and each URL is directed to the same backend server (caching)
fair (third party)	Allocate requests based on the response time of the backend server, with priority given to those with shorter response times.
least_conn	Minimum number of connections, whichever connection has fewer connections will be allocated
custom hash	hash custom key

bash

upstream mamingjuan{
  ip_hash;
  server 127.0.0.1:3000;
}


upstream mamingjuan{
  least_conn;
  server 127.0.0.1:3000;
}


upstream mamingjuan{
  url_hash;
  server 127.0.0.1:3000;
}


upstream mamingjuan{
  fair;
  server 127.0.0.1:3000;
}


upstream mamingjuan{
  hash $request_uri;
  server 127.0.0.1:3000;
}

cache

nginx proxy cache

bash

http{  
    # 缓存路径 目录层级 缓存空间名称和大小 失效时间为7天 最大容量为10g
    proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=cache:100m inactive=60m max_size=10g; 
}

proxy_cache_path cache file path
levels sets the cache file directory level; levels=1:2 represents a two-level directory
keys_zone sets cache name and shared memory size
inactive will be deleted if no one accesses it within the specified time.
max_size is the maximum cache space. If the cache space is full, the resource with the longest cache time will be overwritten by default.

bash

    if ($request_uri ~ ^/cache/(login|logout)) {
      set $nocache 1;
    }
    location / {
       proxy_pass http://mamingjuan;
    }
    location ~ ^/cache/ {
        proxy_cache cache;
        proxy_cache_valid  200 206 304 301 302 60m;   # 对哪些状态码缓存，过期时间为60分钟
        proxy_cache_key $uri;  # 缓存的维度
        proxy_no_cache $nocache;
        proxy_set_header Host $host:$server_port;  # 设置头
        proxy_set_header X-Real-IP $remote_addr;   # 设置头
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;   # 设置头
        proxy_pass http://127.0.0.1:6000;
    }

proxy_cache tmp-test uses the corresponding cache configuration named tmp-test
proxy_cache_valid 200 206 304 301 302 10d; cache httpcode 200... for 10 days
proxy_cache_key $uri defines the unique key of the cache, and performs hash access through the unique key.
proxy_set_header custom http header, used to send to the backend real server.
proxy_pass refers to the path forwarded after proxy. Pay attention to whether the last / is required.

rewrite

URL rewriting and redirection can be achieved.Purposes include URL page jump, compatibility with old versions, SEO optimization (pseudo-static), maintenance (backend maintenance, traffic forwarding), security (pseudo-static)

语法	rewrite regex replacement [flag]
默认	-
上下文	server,location,if

Explanation: regex regular expression refers to the path to be rewritten
replacement The URL to which the target should be replaced
flag identification

Actual combat

bash

rewrite ^(.*)$ /www/reparing.html break;

I won’t go into details about regular expressions, they are almost the same as JavaScript regular expressions.

bash

if($http_user_agent ~ MSIE){
  rewrite ^(.*)$ /msie/$1 break;
}

pcretest

bash

wget https://ftp.pcre.org/pub/pcre/pcre-8.13.tar.gz
tar -xzvf  pcre-8.13.tar.gz
cd pcre-8.13
./configure --enable-utf8  
make
make install
pcretest

flag
The flag bit is the type corresponding to the identification rule.

flag	meaning
last	First match your own location, then create a new request through the rewrite rule and request the server again
break	Match your own location first, then the life cycle will end at the current location, and no subsequent matching will be performed
redirect	Returns 302 redirection yesterday, and will request this server in the future
permanent	Returns 301 permanent redirection, and will directly request the domain name after permanent redirection in the future

bash

location ~ ^/break {
     rewrite ^/break /test break;
     proxy_pass http://127.0.0.1:3000;
}

location ~ ^/last {
      rewrite ^/last /test last;
}

location /test {
      default_type application/json;
      return 200 '{"code":0,"msg":"success"}';
}

curl -vL http://192.168.20.150/redirect

location ~ ^/redirect {
 rewrite ^/redirect http://www.baidu.com redirect;
 rewrite ^/redirect http://www.baidu.com permanent;
}

rewrite priority

First execute the rewrite command in the server
Perform location matching
Then execute rewrite in location

priority in location

The equal sign type (=) has the highest priority.Once a match is successful, no further matches are looked for.
^~type expression.Once a match is successful, no further matches are looked for.
Regular expression types (~ ~*) have the next highest priority.If there are multiple location regular expressions that can match, the one with the longest regular expression will be used.
Regular string match types match by prefix

nginx related knowledge ​

system ​

nginx advantages ​

nginx version ​

Configuration files and uses ​

cgi configuration file ​

Encoding conversion mapping conversion file ​

extension file ​

Daemon management ​

nginx module directory ​

nginx manual and help files ​

nginx service startup management executable file ​

nginx content configuration ​

nginx configuration syntax ​

Global and service configuration ​

event configuration ​

http configuration ​

server configuration ​

Systemd configuration ​

Configuration directory ​

systemctl ​

Start and reload ​

Log related ​

core module ​

Static resource web service ​

CDN ​

Configuration syntax ​

browser cache ​

Cross domain ​

Anti-hotlinking ​

agency service ​

load balancing ​

cache ​

rewrite ​

nginx related knowledge

system

nginx advantages

nginx version

Configuration files and uses

cgi configuration file

Encoding conversion mapping conversion file

extension file

Daemon management

nginx module directory

nginx manual and help files

nginx service startup management executable file

nginx content configuration

nginx configuration syntax

Global and service configuration

event configuration

http configuration

server configuration

Systemd configuration

Configuration directory

systemctl

Start and reload

Log related

core module

Static resource web service

CDN

Configuration syntax

browser cache

Cross domain

Anti-hotlinking

agency service

load balancing

cache

rewrite