[big data practice] flume data collection

flume Quick start summary Apache Flume is a distributed, reliable and available system for effectively collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store. The use of Apache Flume is not limited to log data aggregation. Because the data source is customizable, Flume can be ...

Posted by BigX on Thu, 07 Apr 2022 10:54:03 +0300

INFINI Gateway: Getting Started Guide

Recently, I was lucky to come into contact with the masterpiece of medcl: Limit gateway (INFINI GATEWAY). INFINI GATEWAY has many advantages and many application scenarios. You can Official website Read on. In short, INFINI Gateway is a platform for Elasticsearch High performance application gateway, which contains rich features and is very si ...

Posted by mikem562 on Wed, 06 Apr 2022 09:54:00 +0300

shell script for big data learning

A learning process of cooking chicken and looking for a job preface I am a vegetable chicken looking for big data development post in the future After extensive online search for relevant work materials, it is not difficult to find that companies often need us to have the following skills: 1. Solid SQL foundation and proficient in Hive ...

Posted by RonDahl on Tue, 05 Apr 2022 22:25:38 +0300

[Python reptile series tutorial 22-100] Miss teaches you to climb the audio data of the whole Himalayan station and explore the sounds of nature in the Himalayas

1, Foreword Himalaya FM is a well-known audio sharing platform. Its market share in the mobile audio industry has reached 73%, and the number of users has exceeded 480 million. Today, we will take you to break through the obstacles, explore the sounds of nature in Himalaya, and realize real-time capture and save it locally. Personally, ...

Posted by jefftanner on Tue, 05 Apr 2022 02:03:55 +0300

Hadoop learning road Hadoop cluster construction and simple application

1, Concept understanding Master-slave structure: in a cluster, some nodes will act as the master server, and other servers will act as slave servers. At present, this architecture mode is called master-slave structure. Classification of master-slave structure: 1. One master and many slaves 2. Multi master and multi slave HDFS and YARN in H ...

Posted by fullyscintilla on Mon, 04 Apr 2022 16:37:28 +0300

Flink state programming

summary   the core of Flink processing mechanism is "stateful streaming computing". We have also mentioned "state" many times in previous chapters. Whether it is simple aggregation, window aggregation or the application of processing functions, there will be state. Previously, we have briefly introduced stateful flow p ...

Posted by freakyG on Sun, 03 Apr 2022 13:47:02 +0300

hadoop introduction deployment document

hadoop deployment document Introduction to hadoop What is hadoop 1) Hadoop is a distributed system infrastructure developed by the Apache foundation. 2) It mainly solves the problems of massive data storage and massive data analysis and calculation. 3) In a broad sense, Hadoop usually refers to a broader concept - Hadoop ecosystem. hadoop ...

Posted by php_joe on Fri, 01 Apr 2022 17:37:55 +0300

node connection MongoDB and common operation records

Personal interface for the back end of the whole stack project preparation I think it is necessary to understand some concepts in advance Install mongoDB Library npm i mongodb Install a mongodb Library Connect database // If the database address is not changed, it is the default address const url = "mongodb://localhost:27017"; ...

Posted by goodrunb on Thu, 31 Mar 2022 11:06:18 +0300

[Flink knowledge summary] [Join topic]

Video address: https://www.bilibili.com/video/av92215954/ Document address: https://files.alicdn.com/tpsservice/e4356097e11364edadb5627a892ee53b.pdf Application scenario of Join Exposure related clicks are involved in almost all company apps; Dimension splicing between two streams of data; Widen the watch, etcIn the e-commerce scenario, t ...

Posted by Sharif on Wed, 30 Mar 2022 12:15:51 +0300

day 05 DQL data query language --- connection query --- enter the room

Because sql statements are not case sensitive, all commands in this article use lowercase for convenience Previous contents day 01 first knowledge of Mysql and DDL data definition languageday 02 DML data operation languageday 03 DQL data query language -- a first glimpseday 04 DQL data query language - single table query - a slight succ ...

Posted by kampbell411 on Tue, 29 Mar 2022 07:33:25 +0300