[spring cloud] Hystrix fuse

Hystrix fuse

summary

Problems faced by distributed systems

Problems faced by distributed systems
Applications in complex distributed architecture have dozens of dependencies, and each dependency will inevitably fail at some time

Service avalanche
When calling between multiple microservices, suppose that microservice A calls microservice B and microservice C, and microservice B and microservice C call other microservices, which is the so-called "fan out".
If the fan out link If the response time of A microservice call is too long or unavailable, the call to microservice A will occupy more and more system resources, resulting in system crash, the so-called "avalanche effect".

For high traffic applications, a single back-end dependency may cause all resources on all servers to saturate in a few seconds. Worse than failure, these applications may also lead to increased delays between services, tight backup queues, threads and other system resources, resulting in more cascading failures of the whole system. These indicate the need to isolate and manage failures and delays so that the failure of a single dependency cannot cancel the entire application or system.

Therefore, usually when you find that an instance under a module fails, the module will still receive traffic, and then the problematic module calls other modules, which will lead to cascading failure, or avalanche.

What is it?

Hystrix is an open source library for dealing with delay and fault tolerance of distributed systems. In distributed systems, many dependencies inevitably fail to call, such as timeout and exception. Hystrix can ensure that in the case of a dependency failure, it will not lead to overall service failure, avoid cascading failures, and improve the elasticity of distributed systems.

"Circuit breaker" itself is a kind of switching device. When a service unit fails, through the fault monitoring of the circuit breaker (similar to fusing a fuse), it returns an expected and treatable alternative response (FallBack) to the caller, rather than waiting for a long time or throwing an exception that the caller cannot handle, so as to ensure that the thread of the service caller will not be occupied unnecessarily for a long time, Thus, the spread of faults in the distributed system and even avalanche are avoided.

What can I do

  • service degradation
  • Service fuse
  • Near real-time monitoring

Official website information

https://github.com/Netflix/hystrix/wiki

Hystrix official announcement, stop and enter the dimension

https://github.com/Netflix/hystrix

consequence:

  • Passively fix bugs
  • No more merge requests accepted
  • No new releases

HyStrix key concepts

service degradation

The server is busy, please try again later, don't let the client wait, and immediately return a friendly prompt, fallback

Under what circumstances will a downgrade be issued

  • Abnormal program operation
  • overtime
  • Service fuse triggers service degradation
  • Thread pools / semaphores can also cause service degradation

Service fuse

After the analog fuse reaches the maximum service access, access is directly refused, power is cut off, and then the service degradation method is called and a friendly prompt is returned
It is the fuse: the service is blown - > the service is degraded - > the call link is restored

Service current limiting

Second kill, high concurrency and other operations. It is strictly forbidden to rush over and crowd. Everyone queue up, N per second, in an orderly manner

hystrix case

structure

Create a new cloud provider hystrix payment8001

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-provider-hystrix-payment8001</artifactId>

    <dependencies>
        <!--hystrix-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <!--eureka client-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>
        <dependency>
            <groupId>com.atguigu.springcloud</groupId>
            <artifactId>cloud-api-common</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--monitor-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Hot deployment-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

YML

server:
  port: 8001
spring:
  application:
    name: cloud-provider-hystrix-payment
eureka:
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka

Main start

package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

/**
 * @author zzyy
 * @create 2020/3/6 22:21
 **/
@SpringBootApplication
@EnableEurekaClient
public class PaymentHystrixMain8001 {
    public static void main(String[] args) {
        SpringApplication.run(PaymentHystrixMain8001.class, args);
    }
}

Business class

service

package com.atguigu.springcloud.service;

import org.springframework.stereotype.Service;

import java.util.concurrent.TimeUnit;

/**
 * @author zzyy
 * @create 2020/3/6 22:23
 **/
@Service
public class PaymentService {
    /**
     * Normal access
     *
     * @param id
     * @return
     */
    public String paymentInfo_OK(Integer id) {
        return "Thread pool:" + Thread.currentThread().getName() + " paymentInfo_OK,id:" + id + "\t" + "O(∩_∩)O ha-ha~";
    }

    /**
     * Timeout access
     *
     * @param id
     * @return
     */
    public String paymentInfo_TimeOut(Integer id) {
        int timeNumber = 3;
        try {
            // Pause for 3 seconds
            TimeUnit.SECONDS.sleep(timeNumber);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return "Thread pool:" + Thread.currentThread().getName() + " paymentInfo_TimeOut,id:" + id + "\t" +
                "O(∩_∩)O ha-ha~  time consuming(second)" + timeNumber;
    }
}

controller

package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;
import java.util.concurrent.TimeUnit;

/**
 * @author zzyy
 * @create 2020/3/6 22:30
 **/
@RestController
@Slf4j
public class PaymentController {
    @Resource
    private PaymentService paymentService;

    @Value("${server.port}")
    private String servicePort;

    /**
     * Normal access
     *
     * @param id
     * @return
     */
    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        String result = paymentService.paymentInfo_OK(id);
        log.info("*****result:" + result);
        return result;
    }

    /**
     * Timeout access
     *
     * @param id
     * @return
     */
    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
        String result = paymentService.paymentInfo_TimeOut(id);
        log.info("*****result:" + result);
        return result;

    }
}

Normal test

Start eureka7001

Start Eureka provider hystrix payment8001

visit

Method of success

http://localhost:8001/payment/hystrix/ok/31

Each call takes five seconds

http://localhost:8001/payment/hystrix/timeout/31

All the above module s are OK

Based on the above platform, from right - > error - > degraded fuse - > recovery

High concurrency test

In the case of non high concurrency, but

Jmeter pressure test

Download address
https://jmeter.apache.org/download_jmeter.cgi

Start Jmeter and kill 800 and 120000 requests with 20000 concurrent requests to access paymentInfo_TimeOut service

Another visit

http://localhost:8001/payment/hystrix/timeout/31

Look at the results of the demonstration

Both are in circles

Why did you get stuck?
The default number of working threads of tomcat is full, and there are no extra threads to decompose the pressure and processing

Jmeter pressure measurement conclusion

The above is only the service provider 8001's own test. If the external consumer 80 also visits at this time, the consumer can only wait, which eventually leads to the dissatisfaction of the consumer 80 and the death of the service 8001

Watching the excitement is not too big to abandon. 80 new members joined

cloud-consumer-feign-hystrix-order80

newly build

cloud-consumer-feign-hystrix-order80

POM
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-consumer-feign-hystrix-order80</artifactId>

    <dependencies>
        <!--openfeign-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
        </dependency>
        <!--eureka client-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>
        <dependency>
            <groupId>com.atguigu.springcloud</groupId>
            <artifactId>cloud-api-common</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--monitor-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Hot deployment-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>
YML
server:
  port: 80
eureka:
  client:
    register-with-eureka: false
    fetch-registry: true
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
Main start
package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.openfeign.EnableFeignClients;

/**
 * @author zzyy
 * @date 2020/02/18 17:20
 **/
@SpringBootApplication
@EnableEurekaClient
@EnableFeignClients
public class OrderHystrixMain80 {
    public static void main(String[] args) {
        SpringApplication.run(OrderHystrixMain80.class, args);
    }
}
Business class
PaymentHystrixService
package com.atguigu.springcloud.service;

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

/**
 * @author zzyy
 * @create 2020/3/6 23:19
 **/
@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")
public interface PaymentHystrixService {


    /**
     * Normal access
     *
     * @param id
     * @return
     */
    @GetMapping("/payment/hystrix/ok/{id}")
    String paymentInfo_OK(@PathVariable("id") Integer id);

    /**
     * Timeout access
     *
     * @param id
     * @return
     */
    @GetMapping("/payment/hystrix/timeout/{id}")
    String paymentInfo_TimeOut(@PathVariable("id") Integer id);

}
OrderHyrixController
@RestController
@Slf4j
public class OrderHystrixController {

    @Resource
    private PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id){
        String result = paymentHystrixService.paymentInfo_OK(id);
        return  result;
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    public String paymentInfo_Timeout(@PathVariable("id") Integer id){
        String result = paymentHystrixService.paymentInfo_Timeout(id);
        return  result;
    }
}
Normal test

http://localhost/consumer/payment/hystrix/ok/32

High concurrency test

Open Jmeter to crush 8001 in 20000 concurrent

The address of OK service 8001 visited by consumer 80 micro service

http://localhost/consumer/payment/hystrix/ok/32

Consumer 80,o(╥﹏╥) o

  • Or turn around
  • Or the consumer reports a timeout error

Faults and causes

  • 8001 other interfaces at the same level are trapped because the working threads in the tomcat thread pool have been occupied
  • 80 at this time, call 8001, the client access response is slow, turn around

Above conclusion

It is precisely because of the above failures or poor performance # that our technologies such as degradation / fault tolerance / current limiting were born

How to solve it? Requirements to be solved

Timeout causes the server to slow down (spin)

Timeout no longer wait

Error (downtime or program running error)

Mistakes should be explained

solve

  • The other party's service (8001) has timed out, and the caller (80) cannot be stuck waiting all the time. There must be service degradation
  • The other party's service (8001) is down, and the caller (80) can't wait all the time. There must be service degradation
  • The other party's service (8001) is OK, and the caller (80) has faults or self requirements (his waiting time is less than that of the service provider)

service degradation

Degraded configuration

@HystrixCommand

8001 look for problems from yourself first

Set the peak value of its own call timeout. It can run normally within the peak value, # exceeding the need for a thorough method to deal with, and make a service degradation fallback

8001fallback

Business class enable

@How to handle the exception reported by HystrixCommand
Once calling the service method fails and an error message is thrown, it will automatically call the specified method in the fallbckMethod calling class marked by @ HystrixCommand
@HystrixCommand(fallbackMethod = "paymentInfo_TimeoutHandler",commandProperties = {
        @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds",value = "3000")
})
public String paymentInfo_Timeout(Integer id) {
    int timeNumber = 5;
    try {
        TimeUnit.SECONDS.sleep(timeNumber);
    } catch (Exception e){
         e.printStackTrace();
    }
    return "Thread pool: " + Thread.currentThread().getName()
            + "   paymentInfo_OK,id:" + id + " time consuming(second):" + timeNumber;
}

public String paymentInfo_TimeoutHandler(Integer id) {
    return "/(ToT)/Timeout or exception in calling the payment interface\t" + "\t Current thread pool name" + Thread.currentThread().getName();
}
Illustration

Main startup class activation

@EnableCircuitBreaker

80fallback

80 order micro services can also better protect themselves, and they can also draw gourds for client-side degradation protection

Digression

The hot deployment method we have configured has obvious changes to the java code, but for the modification of the properties in @ HystrixCommand, it is recommended to restart the microservice

POM

<!--hystrix-->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>

YML

server:
  port: 80
eureka:
  client:
    register-with-eureka: false
    fetch-registry: true
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
feign:
  hystrix:
    enabled: true

Main start

@EnableHystrix

Business class

@GetMapping("/consumer/payment/hystrix/timeout/{id}")
@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = {
        @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")
})
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
    //int age = 10/0;
    return paymentHystrixService.paymentInfo_TimeOut(id);
}

public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) {
    return "I'm a consumer 80,The other party's payment system is busy. Please try again in 10 seconds, or your operation is wrong. Please check yourself,o(╥﹏╥)o";
}

Current problems

  • Each business method corresponds to a bottom-up method, and the code expands
  • Separation of unified and customized

terms of settlement

Each method is configured with a??? expand

feign interface series
@DefaultProperties(defaultFallback="")


explain

@DefaultProperties(defaultFallback = ")
1: 1. Each method is configured with a service degradation method. Technically, it is OK. In fact, it is OK
1: It can be used to process individual business results (except for @ default default default) through the core business system

The general and exclusive are separated, which avoids code expansion and reasonably reduces the amount of code. 0 (∩∩) O ha ha~

controller configuration
package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

/**
 * @author zzyy
 * @create 2020/3/6 23:20
 **/
@RestController
@Slf4j
@DefaultProperties(defaultFallback = "payment_Global_FallbackMethod")
public class OrderHystrixController {
    @Resource
    private PaymentHystrixService paymentHystrixService;


       @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        return paymentHystrixService.paymentInfo_OK(id);
    }

       @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    /*@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = {
            @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")
    })*/
    @HystrixCommand
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
        //int age = 10/0;
        return paymentHystrixService.paymentInfo_TimeOut(id);
    }

    public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) {
        return "I'm a consumer 80,The other party's payment system is busy. Please try again in 10 seconds, or your operation is wrong. Please check yourself,o(╥﹏╥)o";
    }

    /**
     * Global fallback
     *
     * @return
     */
    public String payment_Global_FallbackMethod() {
        return "Global Exception handling information,Please try again later.o(╥﹏╥)o";
    }
}

Mixed with business logic??? confusion

When the service is degraded, the client calls the server and the server goes down or shuts down
In this case, the service degradation processing is completed in the client 80, which has nothing to do with the server 8001 # just add an implementation class of service degradation processing to the interface defined by Feign client to realize decoupling
The anomalies we will face in the future
  • function
  • overtime
  • Downtime
Let's look at our business class PaymentController

Modify cloud consumer feign hystrix order80
According to the existing PaymentHystrixService interface of cloud consumer feign hystrix order80, create a new class (PaymentFallbackService) to implement the interface and handle exceptions for the methods in the interface
The PaymentFallbackService class implements the paymentfeinservice interface

YML

Remember to open this annotation

feign:
  hystrix:
    enabled: true

PaymentFeignClientService interface

test

A single eureka starts 7001 first

PaymentHystrixMain8001 start

Normal access test: http://localhost/consumer/payment/hystrix/ok/32

Deliberately shut down microservice 8001

The client calls the prompt itself
At this time, the server provider has been down, but we have degraded the service, # so that the client will get the prompt message when the server is unavailable without hanging up the server

Service fuse

Circuit breaker

In a word, it's the fuse at home

What is a fuse

Overview of fuse mechanism
Fuse mechanism is a microservice link protection mechanism to deal with avalanche effect. When a microservice on the fan out link fails to be available or the response time is too long, the service will be degraded, which will fuse the call of the node microservice and quickly return the wrong response information.
When it is detected that the microservice call response of the node is normal, the call link is restored.

In the Spring Cloud framework, the circuit breaker mechanism is implemented through hystrix. Hystrix will monitor the status of calls between microservices,
When the failed call reaches a certain threshold, the default is 20 calls in 5 seconds, and the fuse mechanism will be started. The annotation of the fuse mechanism is @ HystrixCommand

Great God thesis: https://martinfowler.com/bliki/CircuitBreaker.html

Practical operation

Modify cloud provider hystrix payment8001

PaymentService

// Service fuse
@HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = {
        @HystrixProperty(name = "circuitBreaker.enabled", value = "true"),              //Is the circuit breaker on
        @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"),    //Only after the number of requests reaches
        @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"), //Sleep time window
        @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "60"),  //What is the error rate
})
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
      if(id < 0){
          throw  new RuntimeException("****id Cannot be negative");
      }
      String serialNumber = IdUtil.simpleUUID();

      return  Thread.currentThread().getName() + "\t" + "Call succeeded, serial number:" + serialNumber;
}

public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id){
     return "id Cannot be negative,Please try again later, o(╥﹏╥)o id: " + id;
}
why these parameters

PaymentController

@GetMapping("/payment/circuit/{id}")
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
    String result = paymentService.paymentCircuitBreaker(id);
    log.info("*****result: " + result);
    return result;
}

test

Self test cloud provider hystrix payment8001

correct: http://localhost:8001/payment/circuit/31
Error: http://localhost:8001/payment/circuit/-31

One right and one wrong try try

Key test
Correct for many times, and then correct slowly. It is found that the conditions are not met at the beginning, and even the correct access cannot be carried out

Principle / summary

Great God conclusion


Fuse type

Fuse open

The request no longer calls the current service. The internal setting is generally MTTR (mean time to deal with failure). When the clock set by the long lead is turned on, it will enter the semi fusing state

Fuse off

The service will not be blown after the fuse is closed

Fuse half open

Some requests call the current service according to the rules. If the request is successful and meets the rules, it is considered that the current service is restored to normal and closed

Flow chart of circuit breaker on official website

Official website steps

When does the circuit breaker start to work


Three important parameters of circuit breaker are involved: snapshot time window, threshold value of total requests and threshold value of error percentage.

  1. Snapshot time window: the circuit breaker needs to count some request and error data to determine whether to open, and the statistical time range is the snapshot time window, which defaults to the last 10 seconds.
  2. Threshold of total requests: within the snapshot time window, the threshold of total requests must be met to be eligible for fusing. The default is 20, which means that if the number of calls of the hystrix command is less than 20 within 10 seconds, the circuit breaker will not open even if all requests timeout or fail for other reasons.
  3. Error percentage threshold: when the total number of requests exceeds the threshold within the snapshot time window, for example, 30 calls occur. If timeout exceptions occur in 15 of the 30 calls, that is, more than 50% of the error percentage, the circuit breaker will be opened when the 50% threshold is set by default.
Conditions for opening or closing the circuit breaker
  1. When a certain threshold is met (more than 20 requests in 10 seconds by default)
  2. When the failure rate reaches a certain level (more than 50% of requests in 10 seconds by default)
  3. When the above threshold is reached, the circuit breaker will open
  4. When enabled, all requests will not be forwarded
  5. After a period of time (5 seconds by default), when the circuit breaker is half open, it will allow another request to be forwarded If successful, the circuit breaker will close. If failed, it will continue to open Repeat 4 and 5
After the circuit breaker is opened

  1. When another request is called, the main logic will not be called, but the degraded fallback will be called directly. Through the circuit breaker, it can automatically find errors and switch the degraded logic to the main logic to reduce the response delay.

  2. How to restore the original main logic?
    For this - problem, hystrix also implements the automatic recovery function for us.
    When the circuit breaker is opened and the main logic is fused, hystrix will start a sleep time window. In this time window, the degraded logic is temporary and becomes the main logic. When the sleep time window expires, the circuit breaker will enter the semi open state and release the secondary request to the original main logic. If the request returns normally, the circuit breaker will continue to close and the main logic will recover. If there is still a problem with this request, The circuit breaker continues to enter the open state, and the sleep time window is re timed.

ALI configuration




Service current limiting

In the later advanced chapter, we will explain the Sentinel description of alibaba

hystrix workflow

https://github.com/Netflix/Hystrix/wiki/How-it-Works

Hystrix workflow

Official website legend

Step description

Service monitoring hystrixDashboard

summary

In addition to isolating the calls of dependent services, hystrix also provides quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated through hystrix and display it to users in the form of statistical reports and graphics, including how many requests are executed, how many successes, how many failures, etc. Netflix monitors the above indicators through the hystrix - metrics event stream project. Spring Cloud also provides the integration of the Hystrix Dashboard to transform the monitoring content into a visual interface.

Instrument cluster 9001

Create a new cloud consumer hystrix dashboard9001

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId>
    <description>hystrix monitor</description>


    <dependencies>
        <!--hystrix dashboard-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
        </dependency>
        <!--monitor-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Hot deployment-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

YML

server:
  port: 9001

HystrixDashboardMain9001 + new annotation @ EnableHystrixDashboard

package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;

/**
 * @author zzyy
 * @create 2020/3/7 17:27
 **/
@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardMain9001 {
    public static void main(String[] args) {
        SpringApplication.run(HystrixDashboardMain9001.class);
    }
}

All Provider microservice providing classes (8001 / 8002 / 8003) need to monitor dependency deployment

<!--monitor-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Start cloud consumer hystrix dashboard9001. The micro service will monitor the micro service 8001 later

http://localhost:9001/hystrix

Circuit breaker demonstration (service monitoring hystrixDashboard)

Modify cloud provider hystrix payment8001

Note: the new version of Hystrix needs to specify the monitoring path in the main startup MainAppHystrix8001

package com.atguigu.springcloud;

import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;
import org.springframework.context.annotation.Bean;

/**
 * @author zzyy
 * @create 2020/3/7 17:27
 **/
@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardMain9001 {
    public static void main(String[] args) {
        SpringApplication.run(HystrixDashboardMain9001.class);
    }

    /**
     * This configuration is configured for service monitoring. It has nothing to do with the service fault tolerance itself. It is a problem after the upgrade of springCloud
     * ServletRegistrationBean Because the default path of springboot is not / hystrix stream
     * Just configure the following servlet s in your project
     * @return
     */
    @Bean
    public ServletRegistrationBean getServlet(){
        HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet();
        ServletRegistrationBean<HystrixMetricsStreamServlet> registrationBean = new ServletRegistrationBean<>(streamServlet);
        registrationBean.setLoadOnStartup(1);
        registrationBean.addUrlMappings("/hystrix.stream");
        registrationBean.setName("HystrixMetricsStreamServlet");
        return registrationBean;
    }
}

Unable to connect to Command Metric Stream.

404

Monitoring test

Start one eureka or three eureka clusters

Observation monitoring window

9001 monitoring 8001

Fill in the monitoring address
http://localhost:8001/hystrix.stream

Test address

http://localhost:8001/payment/circuit/31
http://localhost:8001/payment/circuit/-31
The above test passed
ok

First visit the correct address, then the wrong address, and then the correct address, you will find that the icon circuit breakers are slowly released

  • Monitoring results, successful

  • Monitoring result, failed

What do you think?
7 colors

1 turn

Filled circle: there are two meanings. It represents the health degree of the instance through the change of color, and its health degree decreases from green < yellow < orange < red.
In addition to the change of color, the size of the solid circle will also change according to the request flow of the instance. The larger the flow, the larger the solid circle. Therefore, through the display of the solid circle, we can quickly find fault cases and high pressure cases in a large number of examples.

Line 1

Curve: it is used to record the relative change of flow within 2 minutes. It can be used to observe the rising and falling trend of flow.

Description of the whole drawing


Description of the whole drawing 2

Only by understanding one can we understand complex problems

Tags: Spring Cloud

Posted by jakeklem on Sat, 21 May 2022 21:32:40 +0300

Hot Tags

Java Python Javascript Algorithm Front-end Linux C++ data structure Spring Back-end Spring Boot MySQL Database C Android Design Pattern Vue Vue.js Docker leetcode Go Machine Learning Big Data programming language Deep Learning Operation & Maintenance Redis Interview Programming Kubernetes html AI C# SQL css
©2023 Algorithm Zone