c language implementation of queue

Sequential implementation of queues typedef struct{ int data[MaxSize]; int front,rear; } SqQueue; For the initialization of the sequence queue, front points to the queue head element, and rear points to the next position of the queue tail element, that is, the next position that should be inserted. void Init(SqQueue q){ q.front=0 ...

Posted by natbrazil on Fri, 01 Jul 2022 21:46:43 +0300

Big data practice: flink e-commerce user behavior analysis, real-time hot commodity statistics

1. Introduction The first thing to realize is real-time popular commodity statistics. We will analyze it based on UserBehavior data set. The main body of the project is written in Scala, using IDEA as the development environment for project writing, and maven as the project construction and management tool. First, we need to build the project f ...

Posted by The Chancer on Tue, 24 May 2022 09:56:15 +0300

spark learning -- spark SQL

Spark SQL Spark SQL is a module that Spark uses to process structured data. It provides a programming abstraction called DataFrame and acts as a distributed SQL query engine. In Hive, Hive SQL is converted into MapReduce and then submitted to the cluster for execution, which greatly simplifies the complexity of writing MapReduce programs, bec ...

Posted by SoaringUSAEagle on Mon, 23 May 2022 18:39:54 +0300

Big data acquisition case: Python web crawler instance

Web crawler: Web crawler (also known as web page) spider , network robot, in FOAF In the middle of the community, more often referred to as web page chaser), it is a kind of automatic crawling according to certain rules web A program or script that contains information. Other names that are not often used are Ants , automatic indexing, emulator ...

Posted by lobobr on Mon, 23 May 2022 13:35:16 +0300

CDH6.3.1 - installation steps

CDH6.3.1 - installation steps Note - all host passwords should be consistent Prepare installation package MySQL5.7 mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar MySQL driver package mysql-connector-java.jar Cloudera manager package cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cloud ...

Posted by jjmusicpro on Mon, 23 May 2022 10:17:39 +0300

ZooKeeper configuration under Hadoop cluster

Install zookeeper environment zookeeper installation package: https://pan.baidu.com/s/1fpdBs8kbjPj5rlrwusv1iw Extraction code: h1wv jdk environment to be prepared: Reference: https://blog.csdn.net/weixin_44147632/article/details/107796624 Decompression: tar -zxf zookeeper-3.4.5-cdh5 14.2. tar. gz -C /opt/bigdata/hadoop/ Renamed: MV zookeeper- ...

Posted by madmindz on Sun, 22 May 2022 16:24:13 +0300

Correctly use Flink broadcast stream and record the failure of Flink to make checkpoints

Recently, when I was working on a project, I was involved in such a scenario that a relatively small table that will not be changed often should be used as a dimension table to match with the real-time flow. This table is a table in MySQL. My first reaction is to read this table for broadcasting. Inelegant use of broadcast streams The brief c ...

Posted by omanush on Sat, 21 May 2022 23:43:47 +0300

Big data Hadoop learning -- page ranking PageRank algorithm

1, Algorithm description PageRank is web page ranking, also known as page ranking (SOCIAL). Some basic concepts: 1. Web page entering the chain: that is, voting. Hyperlinks to other web pages in the web page are used as other web pages entering the chain, which is equivalent to voting for other web pages; 2. Number of links: if a web page ob ...

Posted by mblack0508 on Sat, 21 May 2022 16:13:37 +0300

Hadoop case wordCount execution process

mapreudce operation Count occurrences of words in a file Put the mapreduce program into a jar package and put it on the hadoop machine Execute hadoop jar xsf.jar mapreduce.Dirverx /123 /usr/local/server/hadoop-2.10.0/out mapreduce.Dirverx is the fully qualified class name of the driver class /123: default read from hdfs for input file path /us ...

Posted by sribala on Sat, 21 May 2022 09:25:07 +0300

Big data - MySQL system information function

1. Function to obtain MySQL version number, number of connections and database name (1)version() version() returns a string indicating the MySQL server version, which uses the utf8 character set. [example] check the current MySQL version number. The SQL statement is as follows: mysql> select version(); +-----------+ | version() | +-------- ...

Posted by galvin on Sat, 21 May 2022 07:51:54 +0300