Sedona spatial data visualization source code analysis

This article is automatically published through MetaWeblog, the original text and updated links: https://extendswind.top/posts/technical/sedona_spatial_big_data_visualization Sedona (GeoSpark) spatial data visualization process is not too complicated, mainly the mapping of each spatial object to the corresponding raster space, similar to vect ...

Posted by lisa99 on Wed, 12 Oct 2022 12:02:28 +0300

201_Spark installation and deployment: Standalone mode

1. Experiment Description Install Spark cluster in spark Standalone run modeExperiment time: 45 minutesThe main steps: Unzip and install SparkAdd Spark configuration fileStart the Spark clusterrun test cases 2. Experimental environment Number of virtual machines: 3 (one master and two slaves, the host names are: master, slave01, slave02 ...

Posted by mnick on Sat, 08 Oct 2022 00:16:10 +0300

JAVA wheel making - zookeeper node operation tool class

ZooKeeper is a distributed, open-source distributed application coordination service. It is an open-source implementation of Chubby of Google and an important component of Hadoop and Hbase. It is a software that provides consistency services for distributed applications. Its functions include configuration maintenance, domain name service, dist ...

Posted by wscreate on Tue, 24 May 2022 06:07:13 +0300

VMware creates hadoop clusters from scratch

VMware creates hadoop clusters from scratch 1. Preparation of template virtual machine environment 1) Prepare a template virtual machine Hadoop 100. The virtual machine configuration requirements are as follows: Note: the Linux system environment in this paper is illustrated by CentOS-7.5-x86-1804 Template virtual machine: 4G memory and 50G har ...

Posted by sunwukung on Mon, 23 May 2022 02:40:46 +0300

ZooKeeper configuration under Hadoop cluster

Install zookeeper environment zookeeper installation package: https://pan.baidu.com/s/1fpdBs8kbjPj5rlrwusv1iw Extraction code: h1wv jdk environment to be prepared: Reference: https://blog.csdn.net/weixin_44147632/article/details/107796624 Decompression: tar -zxf zookeeper-3.4.5-cdh5 14.2. tar. gz -C /opt/bigdata/hadoop/ Renamed: MV zookeeper- ...

Posted by madmindz on Sun, 22 May 2022 16:24:13 +0300

Big data Hadoop learning -- page ranking PageRank algorithm

1, Algorithm description PageRank is web page ranking, also known as page ranking (SOCIAL). Some basic concepts: 1. Web page entering the chain: that is, voting. Hyperlinks to other web pages in the web page are used as other web pages entering the chain, which is equivalent to voting for other web pages; 2. Number of links: if a web page ob ...

Posted by mblack0508 on Sat, 21 May 2022 16:13:37 +0300

Hadoop case wordCount execution process

mapreudce operation Count occurrences of words in a file Put the mapreduce program into a jar package and put it on the hadoop machine Execute hadoop jar xsf.jar mapreduce.Dirverx /123 /usr/local/server/hadoop-2.10.0/out mapreduce.Dirverx is the fully qualified class name of the driver class /123: default read from hdfs for input file path /us ...

Posted by sribala on Sat, 21 May 2022 09:25:07 +0300

Hadoop-day01_(java code simulates hadoop storage data)

hadoop file segmentation idea Requirement: Count the number of people in each class in the text file (total to countless people) 1500100129,Rong Jinan,23,Female,Liberal Arts Class Three 1500100130,Ning Huailian,21,Female,Science fourth class 1500100131,Hu Haoming,22,male,Sixth class of liberal arts 1500100132,Zeng Anhan,22,Female,Fifth class of ...

Posted by Clarkey Boy on Fri, 20 May 2022 20:18:06 +0300

docker builds hadoop cluster (distributed and fully distributed)

Chapter 1 is written in front and must be read 1.1 brief description of Hadoop ecology Note: hadoop is just a platform for storing data. mapreduce is a computing framework, which requires programmers to write programs to process data. Then hadoop is an ecosystem, that is, it also runs HBase database, sqoop, shark and other tools, so as to make ...

Posted by dvt85 on Fri, 20 May 2022 17:54:54 +0300

vivo 10000 large-scale HDFS cluster upgrade HDFS 3 X practice

vivo Internet big data team Lv Jia Hadoop 3. The first stable version of X was released at the end of 2017, with many significant improvements. In terms of HDFS, it supports new features such as error coding, More than 2 NameNodes, router based Federation, Standby NameNode Read, FairCallQueue and intra datanode balancer. These new features bri ...

Posted by mybluehair on Mon, 16 May 2022 05:16:17 +0300