Viewing successfully submitted offset indicator in Flink WebUI is negative

1. Problems After submitting a flink task, I went to WebUi to check the offset indicator, and found that the value turned out to be a negative number (as shown below): The definition given on the official website of this indicator is: For each partition, the offset of the last successful submission to Kafka; so this value is not normal anyway. ...

Posted by jefrat72 on Wed, 18 May 2022 15:44:05 +0300

SOA distributed software architecture combined with Domain Driven Design

1, Definition of SOA and DDD SOA and DDD are common system architectures, but the core of them is different. SOA (service-Oriented Architecture) was proposed by Gartner in 1996. It is a distributed software architecture. It can deploy, combine and use loosely coupled coarse-grained application components through the network according to requi ...

Posted by marcela1637 on Wed, 18 May 2022 13:32:18 +0300

Debezium specifies binlog location to initialize exception troubleshooting and repair

Abnormal phenomenon Recently, the project needs to use Debezium to specify the binlog location to read data. When FileDatabaseHistory is configured to save offset, an unrecognized schema exception will appear. 14:52:18.237 [blc-9.135.12.10:3307] ERROR io.debezium.connector.mysql.MySqlStreamingChangeEventSource - Encountered change event ' ...

Posted by jasonscherer on Wed, 18 May 2022 01:40:00 +0300

How to get Docker images to take off, so simple to set up

I. Introduction Docker is great to use, especially when it comes to DevOps practices. However, when you pull mirrors in China or locally, you often encounter various "constipation" - either the mirror pull is slow, or the connection is intermittent, or the connection times out! When our images are large (like someone dropped a Warcr ...

Posted by genom on Tue, 17 May 2022 13:01:00 +0300

02. Implementation principle and source code analysis of Flink Client

Personal official account: Implementation principle of Flink Client Before we interpret the source code of Flink Client, we must first understand its implementation principle. Main functions of Flink Client We can see that the FLink Client mainly has three tasks. The first is to run the Application, the second is to operate and ma ...

Posted by statrat on Tue, 17 May 2022 00:17:30 +0300

Financial Computing and Programming matplotlib Preliminary

Here is the second lesson in Financial Computing and Programming What I'm learning today is matplotlib - the most basic drawing library in python. import matplotlib.pyplot as plt import numpy as np 1. We first generate a one-dimensional y (to draw a line chart) Just draw a picture np.random.seed(250) y = np.random.standard_normal(20) x ...

Posted by gfmitchell on Mon, 16 May 2022 09:22:37 +0300

Go launched http2 0 request flow analysis (later) - header compression

Reading suggestions This is HTTP 2 0 series, the author recommends reading in the following order: HTTP request in Go -- http1 1 request process analysis Go launched http2 0 request process analysis (previous) Go launched http2 0 request flow analysis (Part 2) -- data frame & flow control review In the previous article (* http2clientco ...

Posted by brandone on Mon, 16 May 2022 09:17:40 +0300

vivo 10000 large-scale HDFS cluster upgrade HDFS 3 X practice

vivo Internet big data team Lv Jia Hadoop 3. The first stable version of X was released at the end of 2017, with many significant improvements. In terms of HDFS, it supports new features such as error coding, More than 2 NameNodes, router based Federation, Standby NameNode Read, FairCallQueue and intra datanode balancer. These new features bri ...

Posted by mybluehair on Mon, 16 May 2022 05:16:17 +0300

flink checkpoint source code analysis II

Overview: the last article generally received the overall concept and process of source code analysis of flycheckpoint. Combined with the code, it introduces the initiation and task execution process of checkpoint Detailed reference: https://blog.csdn.net/weixin_40809627/article/details/108537480 This article will follow the previous article a ...

Posted by freddyw on Sun, 15 May 2022 15:26:01 +0300

Cool interface and powerful function! This Linux performance real-time monitoring tool is super easy to use! Old Smith, do it!

For the maintenance and management of Linux system, its performance monitoring is very important, especially the real-time monitoring data. This data is helpful for us to judge the load pressure of the server, adjust the resource allocation in time, and better serve the business. So today, the migrant worker brother gives you a Linux performanc ...

Posted by pido on Sun, 15 May 2022 13:19:02 +0300