Hystrix fuse
summary
Problems faced by distributed systems
Problems faced by distributed systems
Applications in complex distributed architecture have dozens of dependencies, and each dependency will inevitably fail at some time
Service avalanche
When calling between multiple microservices, suppose that microservice A calls microservice B and microservice C, and microservice B and microservice C call other microservices, which is the so-called "fan out".
If the fan out link If the response time of A microservice call is too long or unavailable, the call to microservice A will occupy more and more system resources, resulting in system crash, the so-called "avalanche effect".
For high traffic applications, a single back-end dependency may cause all resources on all servers to saturate in a few seconds. Worse than failure, these applications may also lead to increased delays between services, tight backup queues, threads and other system resources, resulting in more cascading failures of the whole system. These indicate the need to isolate and manage failures and delays so that the failure of a single dependency cannot cancel the entire application or system.
Therefore, usually when you find that an instance under a module fails, the module will still receive traffic, and then the problematic module calls other modules, which will lead to cascading failure, or avalanche.
What is it?
Hystrix is an open source library for dealing with delay and fault tolerance of distributed systems. In distributed systems, many dependencies inevitably fail to call, such as timeout and exception. Hystrix can ensure that in the case of a dependency failure, it will not lead to overall service failure, avoid cascading failures, and improve the elasticity of distributed systems.
"Circuit breaker" itself is a kind of switching device. When a service unit fails, through the fault monitoring of the circuit breaker (similar to fusing a fuse), it returns an expected and treatable alternative response (FallBack) to the caller, rather than waiting for a long time or throwing an exception that the caller cannot handle, so as to ensure that the thread of the service caller will not be occupied unnecessarily for a long time, Thus, the spread of faults in the distributed system and even avalanche are avoided.
What can I do
- service degradation
- Service fuse
- Near real-time monitoring
Official website information
https://github.com/Netflix/hystrix/wiki
Hystrix official announcement, stop and enter the dimension
https://github.com/Netflix/hystrix
consequence:
- Passively fix bugs
- No more merge requests accepted
- No new releases
HyStrix key concepts
service degradation
The server is busy, please try again later, don't let the client wait, and immediately return a friendly prompt, fallback
Under what circumstances will a downgrade be issued
- Abnormal program operation
- overtime
- Service fuse triggers service degradation
- Thread pools / semaphores can also cause service degradation
Service fuse
After the analog fuse reaches the maximum service access, access is directly refused, power is cut off, and then the service degradation method is called and a friendly prompt is returned
It is the fuse: the service is blown - > the service is degraded - > the call link is restored
Service current limiting
Second kill, high concurrency and other operations. It is strictly forbidden to rush over and crowd. Everyone queue up, N per second, in an orderly manner
hystrix case
structure
Create a new cloud provider hystrix payment8001
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>cloud2020</artifactId> <groupId>com.atguigu.springcloud</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-provider-hystrix-payment8001</artifactId> <dependencies> <!--hystrix--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-hystrix</artifactId> </dependency> <!--eureka client--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId> </dependency> <dependency> <groupId>com.atguigu.springcloud</groupId> <artifactId>cloud-api-common</artifactId> <version>${project.version}</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!--monitor--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Hot deployment--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 8001 spring: application: name: cloud-provider-hystrix-payment eureka: client: register-with-eureka: true fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
Main start
package com.atguigu.springcloud; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; /** * @author zzyy * @create 2020/3/6 22:21 **/ @SpringBootApplication @EnableEurekaClient public class PaymentHystrixMain8001 { public static void main(String[] args) { SpringApplication.run(PaymentHystrixMain8001.class, args); } }
Business class
service
package com.atguigu.springcloud.service; import org.springframework.stereotype.Service; import java.util.concurrent.TimeUnit; /** * @author zzyy * @create 2020/3/6 22:23 **/ @Service public class PaymentService { /** * Normal access * * @param id * @return */ public String paymentInfo_OK(Integer id) { return "Thread pool:" + Thread.currentThread().getName() + " paymentInfo_OK,id:" + id + "\t" + "O(∩_∩)O ha-ha~"; } /** * Timeout access * * @param id * @return */ public String paymentInfo_TimeOut(Integer id) { int timeNumber = 3; try { // Pause for 3 seconds TimeUnit.SECONDS.sleep(timeNumber); } catch (InterruptedException e) { e.printStackTrace(); } return "Thread pool:" + Thread.currentThread().getName() + " paymentInfo_TimeOut,id:" + id + "\t" + "O(∩_∩)O ha-ha~ time consuming(second)" + timeNumber; } }
controller
package com.atguigu.springcloud.controller; import com.atguigu.springcloud.service.PaymentService; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Value; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; import javax.annotation.Resource; import java.util.concurrent.TimeUnit; /** * @author zzyy * @create 2020/3/6 22:30 **/ @RestController @Slf4j public class PaymentController { @Resource private PaymentService paymentService; @Value("${server.port}") private String servicePort; /** * Normal access * * @param id * @return */ @GetMapping("/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id) { String result = paymentService.paymentInfo_OK(id); log.info("*****result:" + result); return result; } /** * Timeout access * * @param id * @return */ @GetMapping("/payment/hystrix/timeout/{id}") public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { String result = paymentService.paymentInfo_TimeOut(id); log.info("*****result:" + result); return result; } }
Normal test
Start eureka7001
Start Eureka provider hystrix payment8001
visit
Method of success
http://localhost:8001/payment/hystrix/ok/31
Each call takes five seconds
http://localhost:8001/payment/hystrix/timeout/31
All the above module s are OK
Based on the above platform, from right - > error - > degraded fuse - > recovery
High concurrency test
In the case of non high concurrency, but
Jmeter pressure test
Download address
https://jmeter.apache.org/download_jmeter.cgi
Start Jmeter and kill 800 and 120000 requests with 20000 concurrent requests to access paymentInfo_TimeOut service
Another visit
http://localhost:8001/payment/hystrix/timeout/31
Look at the results of the demonstration
Both are in circles
Why did you get stuck?
The default number of working threads of tomcat is full, and there are no extra threads to decompose the pressure and processing
Jmeter pressure measurement conclusion
The above is only the service provider 8001's own test. If the external consumer 80 also visits at this time, the consumer can only wait, which eventually leads to the dissatisfaction of the consumer 80 and the death of the service 8001
Watching the excitement is not too big to abandon. 80 new members joined
cloud-consumer-feign-hystrix-order80
newly build
cloud-consumer-feign-hystrix-order80
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>cloud2020</artifactId> <groupId>com.atguigu.springcloud</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-consumer-feign-hystrix-order80</artifactId> <dependencies> <!--openfeign--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-openfeign</artifactId> </dependency> <!--eureka client--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId> </dependency> <dependency> <groupId>com.atguigu.springcloud</groupId> <artifactId>cloud-api-common</artifactId> <version>${project.version}</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!--monitor--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Hot deployment--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 80 eureka: client: register-with-eureka: false fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
Main start
package com.atguigu.springcloud; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; import org.springframework.cloud.openfeign.EnableFeignClients; /** * @author zzyy * @date 2020/02/18 17:20 **/ @SpringBootApplication @EnableEurekaClient @EnableFeignClients public class OrderHystrixMain80 { public static void main(String[] args) { SpringApplication.run(OrderHystrixMain80.class, args); } }
Business class
PaymentHystrixService
package com.atguigu.springcloud.service; import org.springframework.cloud.openfeign.FeignClient; import org.springframework.stereotype.Component; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; /** * @author zzyy * @create 2020/3/6 23:19 **/ @Component @FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT") public interface PaymentHystrixService { /** * Normal access * * @param id * @return */ @GetMapping("/payment/hystrix/ok/{id}") String paymentInfo_OK(@PathVariable("id") Integer id); /** * Timeout access * * @param id * @return */ @GetMapping("/payment/hystrix/timeout/{id}") String paymentInfo_TimeOut(@PathVariable("id") Integer id); }
OrderHyrixController
@RestController @Slf4j public class OrderHystrixController { @Resource private PaymentHystrixService paymentHystrixService; @GetMapping("/consumer/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id){ String result = paymentHystrixService.paymentInfo_OK(id); return result; } @GetMapping("/consumer/payment/hystrix/timeout/{id}") public String paymentInfo_Timeout(@PathVariable("id") Integer id){ String result = paymentHystrixService.paymentInfo_Timeout(id); return result; } }
Normal test
http://localhost/consumer/payment/hystrix/ok/32
High concurrency test
Open Jmeter to crush 8001 in 20000 concurrent
The address of OK service 8001 visited by consumer 80 micro service
http://localhost/consumer/payment/hystrix/ok/32
Consumer 80,o(╥﹏╥) o
- Or turn around
- Or the consumer reports a timeout error
Faults and causes
- 8001 other interfaces at the same level are trapped because the working threads in the tomcat thread pool have been occupied
- 80 at this time, call 8001, the client access response is slow, turn around
Above conclusion
It is precisely because of the above failures or poor performance # that our technologies such as degradation / fault tolerance / current limiting were born
How to solve it? Requirements to be solved
Timeout causes the server to slow down (spin)
Timeout no longer wait
Error (downtime or program running error)
Mistakes should be explained
solve
- The other party's service (8001) has timed out, and the caller (80) cannot be stuck waiting all the time. There must be service degradation
- The other party's service (8001) is down, and the caller (80) can't wait all the time. There must be service degradation
- The other party's service (8001) is OK, and the caller (80) has faults or self requirements (his waiting time is less than that of the service provider)
service degradation
Degraded configuration
@HystrixCommand
8001 look for problems from yourself first
Set the peak value of its own call timeout. It can run normally within the peak value, # exceeding the need for a thorough method to deal with, and make a service degradation fallback
8001fallback
Business class enable
@How to handle the exception reported by HystrixCommand
Once calling the service method fails and an error message is thrown, it will automatically call the specified method in the fallbckMethod calling class marked by @ HystrixCommand
@HystrixCommand(fallbackMethod = "paymentInfo_TimeoutHandler",commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds",value = "3000") }) public String paymentInfo_Timeout(Integer id) { int timeNumber = 5; try { TimeUnit.SECONDS.sleep(timeNumber); } catch (Exception e){ e.printStackTrace(); } return "Thread pool: " + Thread.currentThread().getName() + " paymentInfo_OK,id:" + id + " time consuming(second):" + timeNumber; } public String paymentInfo_TimeoutHandler(Integer id) { return "/(ToT)/Timeout or exception in calling the payment interface\t" + "\t Current thread pool name" + Thread.currentThread().getName(); }
Illustration
Main startup class activation
@EnableCircuitBreaker
80fallback
80 order micro services can also better protect themselves, and they can also draw gourds for client-side degradation protection
Digression
The hot deployment method we have configured has obvious changes to the java code, but for the modification of the properties in @ HystrixCommand, it is recommended to restart the microservice
POM
<!--hystrix--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-hystrix</artifactId> </dependency>
YML
server: port: 80 eureka: client: register-with-eureka: false fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka feign: hystrix: enabled: true
Main start
@EnableHystrix
Business class
@GetMapping("/consumer/payment/hystrix/timeout/{id}") @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500") }) public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { //int age = 10/0; return paymentHystrixService.paymentInfo_TimeOut(id); } public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) { return "I'm a consumer 80,The other party's payment system is busy. Please try again in 10 seconds, or your operation is wrong. Please check yourself,o(╥﹏╥)o"; }
Current problems
- Each business method corresponds to a bottom-up method, and the code expands
- Separation of unified and customized
terms of settlement
Each method is configured with a??? expand
feign interface series
@DefaultProperties(defaultFallback="")
explain
@DefaultProperties(defaultFallback = ")
1: 1. Each method is configured with a service degradation method. Technically, it is OK. In fact, it is OK
1: It can be used to process individual business results (except for @ default default default) through the core business system
The general and exclusive are separated, which avoids code expansion and reasonably reduces the amount of code. 0 (∩∩) O ha ha~
controller configuration
package com.atguigu.springcloud.controller; import com.atguigu.springcloud.service.PaymentHystrixService; import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import lombok.extern.slf4j.Slf4j; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; import javax.annotation.Resource; /** * @author zzyy * @create 2020/3/6 23:20 **/ @RestController @Slf4j @DefaultProperties(defaultFallback = "payment_Global_FallbackMethod") public class OrderHystrixController { @Resource private PaymentHystrixService paymentHystrixService; @GetMapping("/consumer/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id) { return paymentHystrixService.paymentInfo_OK(id); } @GetMapping("/consumer/payment/hystrix/timeout/{id}") /*@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500") })*/ @HystrixCommand public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { //int age = 10/0; return paymentHystrixService.paymentInfo_TimeOut(id); } public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) { return "I'm a consumer 80,The other party's payment system is busy. Please try again in 10 seconds, or your operation is wrong. Please check yourself,o(╥﹏╥)o"; } /** * Global fallback * * @return */ public String payment_Global_FallbackMethod() { return "Global Exception handling information,Please try again later.o(╥﹏╥)o"; } }
Mixed with business logic??? confusion
When the service is degraded, the client calls the server and the server goes down or shuts down
In this case, the service degradation processing is completed in the client 80, which has nothing to do with the server 8001 # just add an implementation class of service degradation processing to the interface defined by Feign client to realize decoupling
The anomalies we will face in the future
- function
- overtime
- Downtime
Let's look at our business class PaymentController
Modify cloud consumer feign hystrix order80
According to the existing PaymentHystrixService interface of cloud consumer feign hystrix order80, create a new class (PaymentFallbackService) to implement the interface and handle exceptions for the methods in the interface
The PaymentFallbackService class implements the paymentfeinservice interface
YML
Remember to open this annotation
feign: hystrix: enabled: true
PaymentFeignClientService interface
test
A single eureka starts 7001 first
PaymentHystrixMain8001 start
Normal access test: http://localhost/consumer/payment/hystrix/ok/32
Deliberately shut down microservice 8001
The client calls the prompt itself
At this time, the server provider has been down, but we have degraded the service, # so that the client will get the prompt message when the server is unavailable without hanging up the server
Service fuse
Circuit breaker
In a word, it's the fuse at home
What is a fuse
Overview of fuse mechanism
Fuse mechanism is a microservice link protection mechanism to deal with avalanche effect. When a microservice on the fan out link fails to be available or the response time is too long, the service will be degraded, which will fuse the call of the node microservice and quickly return the wrong response information.
When it is detected that the microservice call response of the node is normal, the call link is restored.
In the Spring Cloud framework, the circuit breaker mechanism is implemented through hystrix. Hystrix will monitor the status of calls between microservices,
When the failed call reaches a certain threshold, the default is 20 calls in 5 seconds, and the fuse mechanism will be started. The annotation of the fuse mechanism is @ HystrixCommand
Great God thesis: https://martinfowler.com/bliki/CircuitBreaker.html
Practical operation
Modify cloud provider hystrix payment8001
PaymentService
// Service fuse @HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = { @HystrixProperty(name = "circuitBreaker.enabled", value = "true"), //Is the circuit breaker on @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"), //Only after the number of requests reaches @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"), //Sleep time window @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "60"), //What is the error rate }) public String paymentCircuitBreaker(@PathVariable("id") Integer id) { if(id < 0){ throw new RuntimeException("****id Cannot be negative"); } String serialNumber = IdUtil.simpleUUID(); return Thread.currentThread().getName() + "\t" + "Call succeeded, serial number:" + serialNumber; } public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id){ return "id Cannot be negative,Please try again later, o(╥﹏╥)o id: " + id; }
why these parameters
PaymentController
@GetMapping("/payment/circuit/{id}") public String paymentCircuitBreaker(@PathVariable("id") Integer id) { String result = paymentService.paymentCircuitBreaker(id); log.info("*****result: " + result); return result; }
test
Self test cloud provider hystrix payment8001
correct: http://localhost:8001/payment/circuit/31
Error: http://localhost:8001/payment/circuit/-31
One right and one wrong try try
Key test
Correct for many times, and then correct slowly. It is found that the conditions are not met at the beginning, and even the correct access cannot be carried out
Principle / summary
Great God conclusion
Fuse type
Fuse open
The request no longer calls the current service. The internal setting is generally MTTR (mean time to deal with failure). When the clock set by the long lead is turned on, it will enter the semi fusing state
Fuse off
The service will not be blown after the fuse is closed
Fuse half open
Some requests call the current service according to the rules. If the request is successful and meets the rules, it is considered that the current service is restored to normal and closed
Flow chart of circuit breaker on official website
Official website steps
When does the circuit breaker start to work
Three important parameters of circuit breaker are involved: snapshot time window, threshold value of total requests and threshold value of error percentage.
- Snapshot time window: the circuit breaker needs to count some request and error data to determine whether to open, and the statistical time range is the snapshot time window, which defaults to the last 10 seconds.
- Threshold of total requests: within the snapshot time window, the threshold of total requests must be met to be eligible for fusing. The default is 20, which means that if the number of calls of the hystrix command is less than 20 within 10 seconds, the circuit breaker will not open even if all requests timeout or fail for other reasons.
- Error percentage threshold: when the total number of requests exceeds the threshold within the snapshot time window, for example, 30 calls occur. If timeout exceptions occur in 15 of the 30 calls, that is, more than 50% of the error percentage, the circuit breaker will be opened when the 50% threshold is set by default.
Conditions for opening or closing the circuit breaker
- When a certain threshold is met (more than 20 requests in 10 seconds by default)
- When the failure rate reaches a certain level (more than 50% of requests in 10 seconds by default)
- When the above threshold is reached, the circuit breaker will open
- When enabled, all requests will not be forwarded
- After a period of time (5 seconds by default), when the circuit breaker is half open, it will allow another request to be forwarded If successful, the circuit breaker will close. If failed, it will continue to open Repeat 4 and 5
After the circuit breaker is opened
When another request is called, the main logic will not be called, but the degraded fallback will be called directly. Through the circuit breaker, it can automatically find errors and switch the degraded logic to the main logic to reduce the response delay.
How to restore the original main logic?
For this - problem, hystrix also implements the automatic recovery function for us.
When the circuit breaker is opened and the main logic is fused, hystrix will start a sleep time window. In this time window, the degraded logic is temporary and becomes the main logic. When the sleep time window expires, the circuit breaker will enter the semi open state and release the secondary request to the original main logic. If the request returns normally, the circuit breaker will continue to close and the main logic will recover. If there is still a problem with this request, The circuit breaker continues to enter the open state, and the sleep time window is re timed.
ALI configuration
Service current limiting
In the later advanced chapter, we will explain the Sentinel description of alibaba
hystrix workflow
https://github.com/Netflix/Hystrix/wiki/How-it-Works
Hystrix workflow
Official website legend
Step description
Service monitoring hystrixDashboard
summary
In addition to isolating the calls of dependent services, hystrix also provides quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated through hystrix and display it to users in the form of statistical reports and graphics, including how many requests are executed, how many successes, how many failures, etc. Netflix monitors the above indicators through the hystrix - metrics event stream project. Spring Cloud also provides the integration of the Hystrix Dashboard to transform the monitoring content into a visual interface.
Instrument cluster 9001
Create a new cloud consumer hystrix dashboard9001
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>cloud2020</artifactId> <groupId>com.atguigu.springcloud</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId> <description>hystrix monitor</description> <dependencies> <!--hystrix dashboard--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId> </dependency> <!--monitor--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Hot deployment--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 9001
HystrixDashboardMain9001 + new annotation @ EnableHystrixDashboard
package com.atguigu.springcloud; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard; /** * @author zzyy * @create 2020/3/7 17:27 **/ @SpringBootApplication @EnableHystrixDashboard public class HystrixDashboardMain9001 { public static void main(String[] args) { SpringApplication.run(HystrixDashboardMain9001.class); } }
All Provider microservice providing classes (8001 / 8002 / 8003) need to monitor dependency deployment
<!--monitor--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
Start cloud consumer hystrix dashboard9001. The micro service will monitor the micro service 8001 later
http://localhost:9001/hystrix
Circuit breaker demonstration (service monitoring hystrixDashboard)
Modify cloud provider hystrix payment8001
Note: the new version of Hystrix needs to specify the monitoring path in the main startup MainAppHystrix8001
package com.atguigu.springcloud; import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.boot.web.servlet.ServletRegistrationBean; import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard; import org.springframework.context.annotation.Bean; /** * @author zzyy * @create 2020/3/7 17:27 **/ @SpringBootApplication @EnableHystrixDashboard public class HystrixDashboardMain9001 { public static void main(String[] args) { SpringApplication.run(HystrixDashboardMain9001.class); } /** * This configuration is configured for service monitoring. It has nothing to do with the service fault tolerance itself. It is a problem after the upgrade of springCloud * ServletRegistrationBean Because the default path of springboot is not / hystrix stream * Just configure the following servlet s in your project * @return */ @Bean public ServletRegistrationBean getServlet(){ HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet(); ServletRegistrationBean<HystrixMetricsStreamServlet> registrationBean = new ServletRegistrationBean<>(streamServlet); registrationBean.setLoadOnStartup(1); registrationBean.addUrlMappings("/hystrix.stream"); registrationBean.setName("HystrixMetricsStreamServlet"); return registrationBean; } }
Unable to connect to Command Metric Stream.
404
Monitoring test
Start one eureka or three eureka clusters
Observation monitoring window
9001 monitoring 8001
Fill in the monitoring address
http://localhost:8001/hystrix.stream
Test address
http://localhost:8001/payment/circuit/31
http://localhost:8001/payment/circuit/-31
The above test passed
ok
First visit the correct address, then the wrong address, and then the correct address, you will find that the icon circuit breakers are slowly released
Monitoring results, successful
Monitoring result, failed
What do you think?
7 colors
1 turn
Filled circle: there are two meanings. It represents the health degree of the instance through the change of color, and its health degree decreases from green < yellow < orange < red.
In addition to the change of color, the size of the solid circle will also change according to the request flow of the instance. The larger the flow, the larger the solid circle. Therefore, through the display of the solid circle, we can quickly find fault cases and high pressure cases in a large number of examples.
Line 1
Curve: it is used to record the relative change of flow within 2 minutes. It can be used to observe the rising and falling trend of flow.
Description of the whole drawing