SpringBoot + Redis + annotation + interceptor to implement interface idempotency verification

1, Concept

Idempotency, in popular terms, is an interface that initiates the same request multiple times. It must be ensured that the operation can only be executed once
For example:

  • Order interface, cannot create order more than once
  • Payment interface. You can only deduct money once for repeated payment of the same order

  • The callback interface of Alipay may be called back many times, and repeated callbacks must be handled

  • The common form submission interface can only succeed once if you click to submit multiple times due to network timeout and other reasons
    wait

2, Common solutions

  • Unique index – prevent new dirty data
  • token mechanism – prevent repeated submission of pages

  • Pessimistic lock – lock when data is obtained (lock table or lock row)

  • Optimistic lock – it is implemented based on the version number version and verifies the data at the moment of updating the data

  • Distributed lock – redis(jedis, redisson) or zookeeper implementation

  • State machine - state change, judge the state when updating data

3, This paper realizes

In this paper, the second method is adopted, that is, the idempotency verification of the interface is realized through redis + token mechanism

4, Realization idea

To create a unique token for each request that needs to ensure idempotency, first obtain the token and store it in redis. When requesting the interface, put the token in the header or request the interface as a request parameter. The back-end interface determines whether this token exists in redis:

  • If it exists, the business logic is processed normally and the token is deleted from redis. If it is a duplicate request, because the token has been deleted, it cannot pass the verification, and the prompt do not repeat operation is returned
  • If it does not exist, it indicates that the parameter is illegal or the request is repeated, and you can return a prompt

5, Project introduction

  • springboot
  • redis

  • @ApiIdempotent annotation + interceptor intercepts the request

  • @ControllerAdvice global exception handling

  • Pressure measuring tool: jmeter

explain:

This article focuses on the core implementation of idempotence. Details about how springboot integrates redis, ServerResponse, ResponseCode and so on are beyond the scope of this article. Interested partners can check my Github project:

https://github.com/wangzaiplus/springboot/tree/wxw

6, Code implementation
pom

 <!-- Redis-Jedis -->
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9.0</version>
        </dependency>

        <!--lombok Used in this article@Slf4j annotation, It can also be omitted, custom log that will do-->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.16.10</version>
        </dependency>

JedisUtil

package com.wangzaiplus.test.util;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;

@Component
@Slf4j
public class JedisUtil {

    @Autowired
    private JedisPool jedisPool;

    private Jedis getJedis() {
        return jedisPool.getResource();
    }

    /**
     * Set value
     *
     * @param key
     * @param value
     * @return
     */
    public String set(String key, String value) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.set(key, value);
        } catch (Exception e) {
            log.error("set key:{} value:{} error", key, value, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Set value
     *
     * @param key
     * @param value
     * @param expireTime Expiration time in s econds
     * @return
     */
    public String set(String key, String value, int expireTime) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.setex(key, expireTime, value);
        } catch (Exception e) {
            log.error("set key:{} value:{} expireTime:{} error", key, value, expireTime, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Value
     *
     * @param key
     * @return
     */
    public String get(String key) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.get(key);
        } catch (Exception e) {
            log.error("get key:{} error", key, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Delete key
     *
     * @param key
     * @return
     */
    public Long del(String key) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.del(key.getBytes());
        } catch (Exception e) {
            log.error("del key:{} error", key, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Determine whether the key exists
     *
     * @param key
     * @return
     */
    public Boolean exists(String key) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.exists(key.getBytes());
        } catch (Exception e) {
            log.error("exists key:{} error", key, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Set key expiration time
     *
     * @param key
     * @param expireTime Expiration time in s econds
     * @return
     */
    public Long expire(String key, int expireTime) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.expire(key.getBytes(), expireTime);
        } catch (Exception e) {
            log.error("expire key:{} error", key, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    /**
     * Get remaining time
     *
     * @param key
     * @return
     */
    public Long ttl(String key) {
        Jedis jedis = null;
        try {
            jedis = getJedis();
            return jedis.ttl(key);
        } catch (Exception e) {
            log.error("ttl key:{} error", key, e);
            return null;
        } finally {
            close(jedis);
        }
    }

    private void close(Jedis jedis) {
        if (null != jedis) {
            jedis.close();
        }
    }

}

Custom annotation @ ApiIdempotent

package com.wangzaiplus.test.annotation;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Use this annotation on the method of the Controller that needs to ensure the idempotency of the interface
 */
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface ApiIdempotent {
}

Interceptor interceptor

package com.wangzaiplus.test.interceptor;

import com.wangzaiplus.test.annotation.ApiIdempotent;
import com.wangzaiplus.test.service.TokenService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Method;

/**
 * Interface idempotent interceptor
 */
public class ApiIdempotentInterceptor implements HandlerInterceptor {

    @Autowired
    private TokenService tokenService;

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
        if (!(handler instanceof HandlerMethod)) {
            return true;
        }

        HandlerMethod handlerMethod = (HandlerMethod) handler;
        Method method = handlerMethod.getMethod();

        ApiIdempotent methodAnnotation = method.getAnnotation(ApiIdempotent.class);
        if (methodAnnotation != null) {
            check(request);// Idempotency verification: if the verification passes, it will be released; if the verification fails, an exception will be thrown, and a friendly prompt will be returned through unified exception handling
        }

        return true;
    }

    private void check(HttpServletRequest request) {
        tokenService.checkToken(request);
    }

    @Override
    public void postHandle(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, ModelAndView modelAndView) throws Exception {
    }

    @Override
    public void afterCompletion(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, Exception e) throws Exception {
    }
}

TokenServiceImpl

package com.wangzaiplus.test.service.impl;

import com.wangzaiplus.test.common.Constant;
import com.wangzaiplus.test.common.ResponseCode;
import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.exception.ServiceException;
import com.wangzaiplus.test.service.TokenService;
import com.wangzaiplus.test.util.JedisUtil;
import com.wangzaiplus.test.util.RandomUtil;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.text.StrBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import javax.servlet.http.HttpServletRequest;

@Service
public class TokenServiceImpl implements TokenService {

    private static final String TOKEN_NAME = "token";

    @Autowired
    private JedisUtil jedisUtil;

    @Override
    public ServerResponse createToken() {
        String str = RandomUtil.UUID32();
        StrBuilder token = new StrBuilder();
        token.append(Constant.Redis.TOKEN_PREFIX).append(str);

        jedisUtil.set(token.toString(), token.toString(), Constant.Redis.EXPIRE_TIME_MINUTE);

        return ServerResponse.success(token.toString());
    }

    @Override
    public void checkToken(HttpServletRequest request) {
        String token = request.getHeader(TOKEN_NAME);
        if (StringUtils.isBlank(token)) {// token does not exist in header
            token = request.getParameter(TOKEN_NAME);
            if (StringUtils.isBlank(token)) {// There is no token in the parameter
                throw new ServiceException(ResponseCode.ILLEGAL_ARGUMENT.getMsg());
            }
        }

        if (!jedisUtil.exists(token)) {
            throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg());
        }

        Long del = jedisUtil.del(token);
        if (del <= 0) {
            throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg());
        }
    }

}

TestApplication

package com.wangzaiplus.test;

import com.wangzaiplus.test.interceptor.ApiIdempotentInterceptor;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.UrlBasedCorsConfigurationSource;
import org.springframework.web.filter.CorsFilter;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;

@SpringBootApplication
@MapperScan("com.wangzaiplus.test.mapper")
public class TestApplication  extends WebMvcConfigurerAdapter {

    public static void main(String[] args) {
        SpringApplication.run(TestApplication.class, args);
    }

    /**
     * Cross domain
     * @return
     */
    @Bean
    public CorsFilter corsFilter() {
        final UrlBasedCorsConfigurationSource urlBasedCorsConfigurationSource = new UrlBasedCorsConfigurationSource();
        final CorsConfiguration corsConfiguration = new CorsConfiguration();
        corsConfiguration.setAllowCredentials(true);
        corsConfiguration.addAllowedOrigin("*");
        corsConfiguration.addAllowedHeader("*");
        corsConfiguration.addAllowedMethod("*");
        urlBasedCorsConfigurationSource.registerCorsConfiguration("/**", corsConfiguration);
        return new CorsFilter(urlBasedCorsConfigurationSource);
    }

    @Override
    public void addInterceptors(InterceptorRegistry registry) {
        // Interface idempotent interceptor
        registry.addInterceptor(apiIdempotentInterceptor());
        super.addInterceptors(registry);
    }

    @Bean
    public ApiIdempotentInterceptor apiIdempotentInterceptor() {
        return new ApiIdempotentInterceptor();
    }

}

OK, so far, the verification code is ready. Next, test and verify

7, Test verification

1. Get token controller

package com.wangzaiplus.test.controller;

import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.service.TokenService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/token")
public class TokenController {

    @Autowired
    private TokenService tokenService;

    @GetMapping
    public ServerResponse token() {
        return tokenService.createToken();
    }

}

2. TestController, pay attention to the @ ApiIdempotent annotation, and declare this annotation on the method that needs idempotence verification. If it does not need verification, it will have no impact

package com.wangzaiplus.test.controller;

import com.wangzaiplus.test.annotation.ApiIdempotent;
import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.service.TestService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/test")
@Slf4j
public class TestController {

    @Autowired
    private TestService testService;

    @ApiIdempotent
    @PostMapping("testIdempotence")
    public ServerResponse testIdempotence() {
        return testService.testIdempotence();
    }

}

3. Get token

View redis

4. Test interface security: use jmeter test tool to simulate 50 concurrent requests, and take the token obtained in the previous step as a parameter


5. Neither header nor parameter passes token, or the token value is empty, or the token value is filled in indiscriminately, which cannot pass the verification. For example, the token value is "abcd"

8, Attention points (very important)

In the above figure, you can't simply delete the token directly without verifying whether the deletion is successful. There will be a concurrent security problem, because multiple threads may go to line 46 at the same time, and the token has not been deleted at this time, so continue to execute. If you don't verify jedisutil If del (token) is deleted and released directly, the problem of repeated submission will still occur, even if there is only one real deletion operation. Let's repeat it below

Slightly modify the code:


Request again

Look at the console

Although only one token is actually deleted, the deletion result is not verified, so there is still a concurrency problem. Therefore, it must be verified

9, Summary

In fact, the idea is very simple, that is to ensure the uniqueness of each request, so as to ensure the idempotency. Through interceptor + annotation, you don't have to write duplicate code every request. In fact, you can also use spring aop to realize it. It doesn't matter.

Github

https://github.com/wangzaiplus/springboot/tree/wxw

END

Posted by greeneel on Sat, 14 May 2022 00:33:25 +0300