springboot integrates Elasticsearch and uses ElasticSearchTemplate to add, delete, modify and query

Use of ElasticSearchTemplate
ElasticSearchTemplate is more a supplement to ESRepository, which provides some lower level methods.

 

Here are mainly related to some queries, and the same is to build various SearchQuery conditions.
You can also complete the add operation

String documentId = "123456";
        SampleEntity sampleEntity = new SampleEntity();
        sampleEntity.setId(documentId);
        sampleEntity.setMessage("some message");
        IndexQuery indexQuery = new IndexQueryBuilder().withId(sampleEntity.getId()).withObject(sampleEntity).build();
        elasticsearchTemplate.index(indexQuery);

add is mainly done through the index method. You need to build an IndexQuery object

 

To build this Object, you need to set the id, which is the id of your Object. The Object is the Object itself. The indexName and type are declared on your Object javaBean

 

The meaning of other fields can be found by themselves. After the IndexQuery is built, it can be inserted through the index method of the Template.
There are also various deleteIndex, delete, update and other methods in the template. Check them when you use them.
Let's talk about a method of batch insertion. We often need to insert a large amount of test data into ElasticSearch to complete the test search. One by one insertion is certainly not possible. ES provides the function of batch insertion of data - bulk.
As mentioned earlier, JPA's save method can also save (List) batch interpolation, but it is suitable for small amount of data. To complete the insertion of large data, we need to use the bulk provided by ES, which can quickly insert millions of data.

public void bulkIndex(List<IndexQuery> queries) {
        BulkRequestBuilder bulkRequest = this.client.prepareBulk();
        Iterator var3 = queries.iterator();
 
        while(var3.hasNext()) {
            IndexQuery query = (IndexQuery)var3.next();
            bulkRequest.add(this.prepareIndex(query));
        }
 
        BulkResponse bulkResponse = (BulkResponse)bulkRequest.execute().actionGet();
        if (bulkResponse.hasFailures()) {
            Map<String, String> failedDocuments = new HashMap();
            BulkItemResponse[] var5 = bulkResponse.getItems();
            int var6 = var5.length;
 
            for(int var7 = 0; var7 < var6; ++var7) {
                BulkItemResponse item = var5[var7];
                if (item.isFailed()) {
                    failedDocuments.put(item.getId(), item.getFailureMessage());
                }
            }
 
            throw new ElasticsearchException("Bulk indexing has failures. Use ElasticsearchException.getFailedDocuments() for detailed messages [" + failedDocuments + "]", failedDocuments);
        }
    }
 
    public void bulkUpdate(List<UpdateQuery> queries) {
        BulkRequestBuilder bulkRequest = this.client.prepareBulk();
        Iterator var3 = queries.iterator();
 
        while(var3.hasNext()) {
            UpdateQuery query = (UpdateQuery)var3.next();
            bulkRequest.add(this.prepareUpdate(query));
        }
 
        BulkResponse bulkResponse = (BulkResponse)bulkRequest.execute().actionGet();
        if (bulkResponse.hasFailures()) {
            Map<String, String> failedDocuments = new HashMap();
            BulkItemResponse[] var5 = bulkResponse.getItems();
            int var6 = var5.length;
 
            for(int var7 = 0; var7 < var6; ++var7) {
                BulkItemResponse item = var5[var7];
                if (item.isFailed()) {
                    failedDocuments.put(item.getId(), item.getFailureMessage());
                }
            }
 
            throw new ElasticsearchException("Bulk indexing has failures. Use ElasticsearchException.getFailedDocuments() for detailed messages [" + failedDocuments + "]", failedDocuments);
        }
    }

Just like inserting a single piece of data into index, what we need here is List, that's all. Isn't it very simple.

/***
     * Batch insert data
     */
    @GetMapping(value = "/batchInsert")
    public void batchInsert(){

        int counter = 0;

            //judge index Does it exist
            if (!template.indexExists(CAR_INDEX_NAME)) {
                template.createIndex(CAR_INDEX_NAME);
            }

            Gson gson = new Gson();
            List<IndexQuery> queries = new ArrayList<IndexQuery>();
            List<Car> cars = this.assembleTestData();
            if(cars != null && cars.size()>0){
                for (Car car : cars) {
                    IndexQuery indexQuery = new IndexQuery();
                    indexQuery.setId(car.getId().toString());
                    indexQuery.setSource(gson.toJson(car));
                    indexQuery.setIndexName(CAR_INDEX_NAME);
                    indexQuery.setType(CAR_INDEX_TYPE);
                    queries.add(indexQuery);
                    //Batch submit index
                    if (counter % 500 == 0) {
                        template.bulkIndex(queries);
                        queries.clear();
                        System.out.println("bulkIndex counter : " + counter);
                    }
                    counter++;
                }

            }
            //Don't forget to submit the index of insufficient batches at last
            if (queries.size() > 0) {
                template.bulkIndex(queries);
            }
            template.refresh(CAR_INDEX_NAME);
       }

Here, 1 million objects are created and inserted with bulkIndex every 500. The speed is very fast, and millions of data are inserted at the speed of seconds.

OK, this article mainly talks about the usage of ElasticSearchRepository and ElasticSearchTemplate and the way to construct QueryBuilder
``
Reprint: https://blog.csdn.net/tianyaleixiaowu/article/details/76149547

Tags: Java

Posted by loony383 on Wed, 11 May 2022 12:30:36 +0300