shell script for big data learning
A learning process of cooking chicken and looking for a job
preface
I am a vegetable chicken looking for big data development post in the future After extensive online search for relevant work materials, it is not difficult to find that companies often need us to have the following skills: 1. Solid SQL foundation and proficient in Hive ...
Posted by RonDahl on Tue, 05 Apr 2022 22:25:38 +0300
[Python reptile series tutorial 22-100] Miss teaches you to climb the audio data of the whole Himalayan station and explore the sounds of nature in the Himalayas
1, Foreword
Himalaya FM is a well-known audio sharing platform. Its market share in the mobile audio industry has reached 73%, and the number of users has exceeded 480 million. Today, we will take you to break through the obstacles, explore the sounds of nature in Himalaya, and realize real-time capture and save it locally.
Personally, ...
Posted by jefftanner on Tue, 05 Apr 2022 02:03:55 +0300
Hadoop learning road Hadoop cluster construction and simple application
1, Concept understanding
Master-slave structure: in a cluster, some nodes will act as the master server, and other servers will act as slave servers. At present, this architecture mode is called master-slave structure.
Classification of master-slave structure:
1. One master and many slaves
2. Multi master and multi slave
HDFS and YARN in H ...
Posted by fullyscintilla on Mon, 04 Apr 2022 16:37:28 +0300
Flink state programming
summary
the core of Flink processing mechanism is "stateful streaming computing". We have also mentioned "state" many times in previous chapters. Whether it is simple aggregation, window aggregation or the application of processing functions, there will be state. Previously, we have briefly introduced stateful flow p ...
Posted by freakyG on Sun, 03 Apr 2022 13:47:02 +0300
hadoop introduction deployment document
hadoop deployment document
Introduction to hadoop
What is hadoop
1) Hadoop is a distributed system infrastructure developed by the Apache foundation.
2) It mainly solves the problems of massive data storage and massive data analysis and calculation.
3) In a broad sense, Hadoop usually refers to a broader concept - Hadoop ecosystem.
hadoop ...
Posted by php_joe on Fri, 01 Apr 2022 17:37:55 +0300
node connection MongoDB and common operation records
Personal interface for the back end of the whole stack project
preparation
I think it is necessary to understand some concepts in advance
Install mongoDB Library
npm i mongodb
Install a mongodb Library
Connect database
// If the database address is not changed, it is the default address
const url = "mongodb://localhost:27017";
...
Posted by goodrunb on Thu, 31 Mar 2022 11:06:18 +0300
[Flink knowledge summary] [Join topic]
Video address: https://www.bilibili.com/video/av92215954/ Document address: https://files.alicdn.com/tpsservice/e4356097e11364edadb5627a892ee53b.pdf
Application scenario of Join
Exposure related clicks are involved in almost all company apps; Dimension splicing between two streams of data; Widen the watch, etcIn the e-commerce scenario, t ...
Posted by Sharif on Wed, 30 Mar 2022 12:15:51 +0300
day 05 DQL data query language --- connection query --- enter the room
Because sql statements are not case sensitive, all commands in this article use lowercase for convenience
Previous contents
day 01 first knowledge of Mysql and DDL data definition languageday 02 DML data operation languageday 03 DQL data query language -- a first glimpseday 04 DQL data query language - single table query - a slight succ ...
Posted by kampbell411 on Tue, 29 Mar 2022 07:33:25 +0300
The most complete in history - Kafka manager configuration and installation Kerberos (ambari HDP) authentication
This article uses Ambari's kafka to configure kafka manager.
CDH and open source can follow the same steps.
kafka is the to enable kerberos authentication.
Kafka manager function
First, let's take a look at the role of Kafka Manager:
Manage multiple clusters
Easily check cluster status (subject, consumer, offset, ag ...
Posted by jdbfitz on Mon, 28 Mar 2022 09:49:32 +0300
Java master Zhenjing application framework volume: PDF ultra clear version of Java Web core framework
Content introduction
Java master Sutra: Java Web core framework (application framework volume)
Author: edited by Liu zhongbing Java Research Laboratory
[book introduction]
This book first analyzes the hierarchical design method of Java Web application, selects the application framework, and then explains various Java Web application framewo ...
Posted by DoddsAntS on Sun, 27 Mar 2022 21:14:43 +0300