[big data practice] flume data collection
flume
Quick start
summary
Apache Flume is a distributed, reliable and available system for effectively collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.
The use of Apache Flume is not limited to log data aggregation. Because the data source is customizable, Flume can be ...
Posted by BigX on Thu, 07 Apr 2022 10:54:03 +0300
INFINI Gateway: Getting Started Guide
Recently, I was lucky to come into contact with the masterpiece of medcl: Limit gateway (INFINI GATEWAY). INFINI GATEWAY has many advantages and many application scenarios. You can Official website Read on. In short, INFINI Gateway is a platform for Elasticsearch High performance application gateway, which contains rich features and is very si ...
Posted by mikem562 on Wed, 06 Apr 2022 09:54:00 +0300
shell script for big data learning
A learning process of cooking chicken and looking for a job
preface
I am a vegetable chicken looking for big data development post in the future After extensive online search for relevant work materials, it is not difficult to find that companies often need us to have the following skills: 1. Solid SQL foundation and proficient in Hive ...
Posted by RonDahl on Tue, 05 Apr 2022 22:25:38 +0300
[Python reptile series tutorial 22-100] Miss teaches you to climb the audio data of the whole Himalayan station and explore the sounds of nature in the Himalayas
1, Foreword
Himalaya FM is a well-known audio sharing platform. Its market share in the mobile audio industry has reached 73%, and the number of users has exceeded 480 million. Today, we will take you to break through the obstacles, explore the sounds of nature in Himalaya, and realize real-time capture and save it locally.
Personally, ...
Posted by jefftanner on Tue, 05 Apr 2022 02:03:55 +0300
Hadoop learning road Hadoop cluster construction and simple application
1, Concept understanding
Master-slave structure: in a cluster, some nodes will act as the master server, and other servers will act as slave servers. At present, this architecture mode is called master-slave structure.
Classification of master-slave structure:
1. One master and many slaves
2. Multi master and multi slave
HDFS and YARN in H ...
Posted by fullyscintilla on Mon, 04 Apr 2022 16:37:28 +0300
Flink state programming
summary
the core of Flink processing mechanism is "stateful streaming computing". We have also mentioned "state" many times in previous chapters. Whether it is simple aggregation, window aggregation or the application of processing functions, there will be state. Previously, we have briefly introduced stateful flow p ...
Posted by freakyG on Sun, 03 Apr 2022 13:47:02 +0300
hadoop introduction deployment document
hadoop deployment document
Introduction to hadoop
What is hadoop
1) Hadoop is a distributed system infrastructure developed by the Apache foundation.
2) It mainly solves the problems of massive data storage and massive data analysis and calculation.
3) In a broad sense, Hadoop usually refers to a broader concept - Hadoop ecosystem.
hadoop ...
Posted by php_joe on Fri, 01 Apr 2022 17:37:55 +0300
node connection MongoDB and common operation records
Personal interface for the back end of the whole stack project
preparation
I think it is necessary to understand some concepts in advance
Install mongoDB Library
npm i mongodb
Install a mongodb Library
Connect database
// If the database address is not changed, it is the default address
const url = "mongodb://localhost:27017";
...
Posted by goodrunb on Thu, 31 Mar 2022 11:06:18 +0300
[Flink knowledge summary] [Join topic]
Video address: https://www.bilibili.com/video/av92215954/ Document address: https://files.alicdn.com/tpsservice/e4356097e11364edadb5627a892ee53b.pdf
Application scenario of Join
Exposure related clicks are involved in almost all company apps; Dimension splicing between two streams of data; Widen the watch, etcIn the e-commerce scenario, t ...
Posted by Sharif on Wed, 30 Mar 2022 12:15:51 +0300
day 05 DQL data query language --- connection query --- enter the room
Because sql statements are not case sensitive, all commands in this article use lowercase for convenience
Previous contents
day 01 first knowledge of Mysql and DDL data definition languageday 02 DML data operation languageday 03 DQL data query language -- a first glimpseday 04 DQL data query language - single table query - a slight succ ...
Posted by kampbell411 on Tue, 29 Mar 2022 07:33:25 +0300