Dubbo source code analysis: Dubbo service export process

Relevant knowledge

Subsequent authors will improve each article in this series and suggest collection

Dubbo URL

When you look at the Dubbo source code, you often see the URL parameter, so share the Dubbo URL first

URL is the address of a resource on the Internet. In Dubbo, it is used as the function of configuring the bus, and the configurations related to services are placed on the bus. A standard URL is as follows

protocol://username:password@host:port/path?key=value&key=value

The constructor of URL object in Dubbo is as follows

public URL(String protocol, String username, String password, String host, int port, String path, Map<String, String> parameters) {
}

The parsing parameters of the URL object in Dubbo are as follows

parameter explain
protocol Various protocols in dubbo, dubbo, thrift, http
username/password User name / password
host/port Host / port
path Path. The default is the interface name. It can be set
parameters Parameter key value pair

Some typical Dubbo URL s are as follows

// Describe a dubbo protocol service
dubbo://192.168.1.6:20880/moe.cnkirito.sample.HelloService?timeout=3000

// Describe a zookeeper registry
zookeeper://127.0.0.1:2181/org.apache.dubbo.registry.RegistryService?application=demo-consumer&dubbo=2.0.2&interface=org.apache.dubbo.registry.RegistryService&pid=1214&qos.port=33333&timestamp=1545721981946

// Describe a consumer
consumer://30.5.120.217/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=consumers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=1209&qos.port=33333&side=consumer&timestamp=1545721827784

In the subsequent introduction, you can see that during the operation of Dubbo, the URL will be frequently modified and added

Why is there a URL object? In fact, it is mainly to facilitate the process of parameter transmission. Put all service related parameters in the URL.

Invoker

You will often see Invoker in Dubbo. You can understand Invoker as an executable

The service consumer is used to perform remote calls (i.e. proxy classes, which encapsulate details such as network communication)
Method used by service provider to call

Let's take a look at the overall process of service export and have a general understanding of the process of service export

XML configuration parsing process

dubbo namespacehandler to parse dubbo's scame and convert it into the corresponding object

We can understand that the application node will be resolved into an ApplicationConfig object, and the service node will be resolved into a ServiceBean object (implementing the ApplicationListener interface), that is, each exported service corresponds to a ServiceBean object

ServiceBean has two important methods

afterPropertiesSet: initializes the properties of some service s
onApplicationEvent: listen to the spring container refresh event and export the service in this method

According to the initialization order of beans, the afterpropertieset method is executed first, and then the onApplicationEvent method is executed

Start service export

ServiceBean implements the ApplicationListener interface

public class ServiceBean<T> extends ServiceConfig<T> implements InitializingBean, DisposableBean,
        ApplicationContextAware, ApplicationListener<ContextRefreshedEvent>, BeanNameAware,
        ApplicationEventPublisherAware {

Therefore, the supervisor starts exporting the service when he hears the ContextRefreshedEvent event

// ServiceBean.java
// Listen to the event and start the service export in observer mode
@Override
public void onApplicationEvent(ContextRefreshedEvent event) {
    // Exported & & canceled export
    if (!isExported() && !isUnexported()) {
        if (logger.isInfoEnabled()) {
            logger.info("The service ready on spring started. service: " + getInterface());
        }
        // Entry method for service export
        export();
    }
}

Then call to

// ServiceConfig.java
public synchronized void export() {
    // Check and update configuration
    checkAndUpdateSubConfigs();

    // When there is a configuration similar to the following, the service will not be exposed, such as local debugging
    // <dubbo:provider export="false" />
    if (!shouldExport()) {
        return;
    }

    // Delayed export service
    if (shouldDelay()) {
        delayExportExecutor.schedule(this::doExport, delay, TimeUnit.MILLISECONDS);
    } else {
        // Export services now
        doExport();
    }
}

When you export the service in doExport, you will call the doExportUrls() method

You can see that a service can be exported in many forms and registered in multiple registries

// ServiceConfig.java
// Multi protocol multi registry export service
// Multiple < Dubbo: Protocol / > are configured in the configuration file
// Multiple < Dubbo: config center > are configured in the configuration file
private void doExportUrls() {
    // Load registry link
    List<URL> registryURLs = loadRegistries(true);
    for (ProtocolConfig protocolConfig : protocols) {
        String pathKey = URL.buildKey(getContextPath(protocolConfig).map(p -> p + "/" + path).orElse(path), group, version);
        // Application model
        // Interface corresponding to the implementation class corresponding to the service name
        ProviderModel providerModel = new ProviderModel(pathKey, ref, interfaceClass);
        ApplicationModel.initProviderModel(pathKey, providerModel);
        // Export services under each agreement and register with each registry
        doExportUrlsFor1Protocol(protocolConfig, registryURLs);
    }
}

The doExportUrlsFor1Protocol method is relatively long, so it is cut into several ends for analysis

// ServiceConfig.java
private void doExportUrlsFor1Protocol(ProtocolConfig protocolConfig, List<URL> registryURLs) {
    String name = protocolConfig.getName();
    if (StringUtils.isEmpty(name)) {
        // The default protocol is dubbo
        name = Constants.DUBBO;
    }
	
	// Omit some codes
	
    String scope = url.getParameter(Constants.SCOPE_KEY);

From the first line to get the scope attribute, this block is mainly to construct the url parameter to see what the final url object looks like


The content of string is as follows

dubbo://192.168.97.70:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bean.name=org.apache.dubbo.demo.DemoService&bind.ip=192.168.97.70&bind.port=20880&default.deprecated=false&default.dynamic=false&default.register=true&deprecated=false&dubbo=2.0.2&dynamic=false&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=3752&qos.port=22222&register=true&release=&side=provider&timestamp=1599285134651

Now we will officially start the process of service exposure. We can export the following three types of services according to the configuration

<!-- Export local service -->
<dubbo:service scope="local" />
<!-- Export remote services -->
<dubbo:service scope="remote" />
<!-- Do not export services -->
<dubbo:service scope="none" />

Export local service

// ServiceConfig.java
private void exportLocal(URL url) {
    if (!Constants.LOCAL_PROTOCOL.equalsIgnoreCase(url.getProtocol())) {
        // Displays the specified injvm protocol for exposure
        URL local = URLBuilder.from(url)
                .setProtocol(Constants.LOCAL_PROTOCOL)
                .setHost(LOCALHOST_VALUE)
                .setPort(0)
                .build();
        // InjvmProtocol#export will be called here
        // Return to InjvmExporter
        Exporter<?> exporter = protocol.export(
                proxyFactory.getInvoker(ref, (Class) interfaceClass, local));
        exporters.add(exporter);
        logger.info("Export dubbo service " + interfaceClass.getName() + " to local registry");
    }
}

Look at the definitions of protocol and proxyFactory member variables, which are to obtain adaptive extension classes.
That is, the framework helps us generate the proxy class. The proxy class obtains the corresponding value from the url during execution, and then returns the corresponding implementation class

private static final Protocol protocol = ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();

private static final ProxyFactory proxyFactory = ExtensionLoader.getExtensionLoader(ProxyFactory.class).getAdaptiveExtension();

The old rule is to use Arthas to look at the generated proxy class, the typical Dubbo SPI code

curl -O https://alibaba.github.io/arthas/arthas-boot.jar
java -jar arthas-boot.jar
# Select the incoming process according to the previous sequence number, and then execute the following command
jad *Adaptive
jad org.apache.dubbo.rpc.Protocol$Adaptive

The generated code is as follows. I won't look at the generated proxy class in the subsequent steps. It's all a routine

public class Protocol$Adaptive implements Protocol {
    public Exporter export(Invoker invoker) throws RpcException {
        String string;
        if (invoker == null) {
            throw new IllegalArgumentException("org.apache.dubbo.rpc.Invoker argument == null");
        }
        if (invoker.getUrl() == null) {
            throw new IllegalArgumentException("org.apache.dubbo.rpc.Invoker argument getUrl() == null");
        }
        URL uRL = invoker.getUrl();
        String string2 = string = uRL.getProtocol() == null ? "dubbo" : uRL.getProtocol();
        if (string == null) {
            throw new IllegalStateException(new StringBuffer().append("Failed to get extension (org.apache.dubbo.rpc.Protocol) name from url (").append(uRL.toString()).append(") use keys([protocol])").toString());
        }
        Protocol protocol = (Protocol)ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(string);
        return protocol.export(invoker);
    }

    public Invoker refer(Class class_, URL uRL) throws RpcException {
        String string;
        if (uRL == null) {
            throw new IllegalArgumentException("url == null");
        }
        URL uRL2 = uRL;
        String string2 = string = uRL2.getProtocol() == null ? "dubbo" : uRL2.getProtocol();
        if (string == null) {
            throw new IllegalStateException(new StringBuffer().append("Failed to get extension (org.apache.dubbo.rpc.Protocol) name from url (").append(uRL2.toString()).append(") use keys([protocol])").toString());
        }
        Protocol protocol = (Protocol)ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(string);
        return protocol.refer(class_, uRL);
    }

    public void destroy() {
        throw new UnsupportedOperationException("The method public abstract void org.apache.dubbo.rpc.Protocol.destroy() of interface org.apache.dubbo.rpc.Protocol is not adaptive method!");
    }

    public int getDefaultPort() {
        throw new UnsupportedOperationException("The method public abstract int org.apache.dubbo.rpc.Protocol.getDefaultPort() of interface org.apache.dubbo.rpc.Protocol is not adaptive method!");
    }
}

As you can see from the code

When the protocol in the URL is registry, the implementation class of the protocol is RegistryProtocol
When the protocol in the URL is injvm, the implementation class of the protocol is InjvmProtocol

Carefully analyze the execution process of this line of code

// ServiceConfig.java
Exporter<?> exporter = protocol.export(
        proxyFactory.getInvoker(ref, (Class) interfaceClass, local));

JavassistProxyFactory wraps the service object as AbstractProxyInvoker and then exports it as InjvmExporter by InjvmProtocol#export

Export AbstractProxyInvoker as InjvmExporter

The main purpose of packaging Invoker as Exporter is to facilitate the management of the lifecycle of Invoker

getInvoker method of JavassistProxyFactory

public class JavassistProxyFactory extends AbstractProxyFactory {

    /**
     * For the provider side, wrap the service object into an Invoker object
     */
    @Override
    public <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) {
        // TODO Wrapper cannot handle this scenario correctly: the classname contains '$'
        final Wrapper wrapper = Wrapper.getWrapper(proxy.getClass().getName().indexOf('$') < 0 ? proxy.getClass() : type);
        // Override the doInvoke method of the AbstractProxyInvoker class
        return new AbstractProxyInvoker<T>(proxy, type, url) {
            @Override
            protected Object doInvoke(T proxy, String methodName,
                                      Class<?>[] parameterTypes,
                                      Object[] arguments) throws Throwable {
                // This method calls and executes the local method
                return wrapper.invokeMethod(proxy, methodName, parameterTypes, arguments);
            }
        };
    }

}

When there is a service call, the AbstractProxyInvoker#doInvoke will be executed (the process of service call will be introduced in the subsequent articles), and the wrapper#invokeMethod will be executed in this method. Wrapper is a class generated by the framework with Javassist, which wraps the implementation class of the service, mainly to reduce reflection calls.


You can take a look at the JdkProxyFactory#getInvoker (another way to generate an Invoker) method. It reflects the execution method directly according to the call information, which is inefficient

Summarize the process of exporting local services.

Export remote services

When looking at the following code, I still need to mention that there are two types of URL s when exporting remote services

  1. Application type, such as dubbo://192.168.97.70:20880/org.apache.dubbo.demo.DemoService
  2. Type of registry, e.g registry://127.0.0.1:2181/com.alibaba.dubbo.registry.RegistryService , zookeeper://127.0.0.1:2181/com.alibaba.dubbo.registry.RegistryService

Let's mainly look at the process from Invoker to Exporter. Other processes are similar to the process of exporting local services

// ServiceConfig.java
// Start exporting services here
// RegistryProtocol called
Exporter<?> exporter = protocol.export(wrapperInvoker);

The wrapperInvoker is passed in. If we trace the content in the url, we can see the final generated class

Protocol is registry, so the class selected by protocol at this time is RegistryProtocol, and then follow RegistryProtocol

After this method is executed, the whole process of service export will be completed. Focus on the three red parts in the figure

URL registryUrl = getRegistryUrl(originInvoker);

What this line of code does is simple

  1. From the parameters, the key is the registry, and the value is zookeeper. Change the protocol to zookeeper
  2. Remove the registry in parameters

Just compare it with the previous picture

Because the protocol of registryUrl = zookeeper, the Registry created later is ZookeeperRegistry

The process of exporting services is as follows:,

// RegistryProtocol.java
// Export service, service started
final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker, providerUrl);

The service startup link is relatively long, so a separate section is opened for analysis

// Register services with the registry
register(registryUrl, registeredProviderUrl);

After registration, the method returns to DestroyableExporter, and the export is completed.

Draw a picture to summarize

Service startup process

// RegistryProtocol.java
private <T> ExporterChangeableWrapper<T> doLocalExport(final Invoker<T> originInvoker, URL providerUrl) {
    // Get the key of service cache
    String key = getCacheKey(originInvoker);

    // A new feature of java8, which determines whether the specified key exists. If it does not exist, it will call the function interface, calculate the value and put it into the map
    return (ExporterChangeableWrapper<T>) bounds.computeIfAbsent(key, s -> {
        Invoker<?> invokerDelegete = new InvokerDelegate<>(originInvoker, providerUrl);
        return new ExporterChangeableWrapper<>((Exporter<T>) protocol.export(invokerDelegete), originInvoker);
    });
}

Because protocol The final implementation class of export (invokerdelegate) is determined by the protocol in providerUrl. Let's see what it is?

protocol=dubbo, so the DubboProtocol#export method will be executed to return an Exporter

This part has an important part

protocol.export(invokerDelegete)

When using SPI to obtain a specific protocol, it will be wrapped by two wrapper classes (Dubbo SPI said this feature)
ProtocolListenerWrapper and ProtocolFilterWrapper

Therefore, when calling the implementation of various protocols, the call link is as follows

ProtocolListenerWrapper: Protocol listener
ProtocolFilterWrapper: executes various filters
QosProtocolWrapper: online operation and maintenance service

// DubboProtocol.java
@Override
public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException {
    URL url = invoker.getUrl();

    // export service.
    // Get the key of the service
    // For example, org apache. demo. DemoService:20880
    // Convert the Invoker into an Exporter and save it in the exporterMap
    String key = serviceKey(url);
    DubboExporter<T> exporter = new DubboExporter<T>(invoker, key, exporterMap);
    exporterMap.put(key, exporter);

	// Omit some codes

    // Different service exports of the same machine will only open one NettyServer
    openServer(url);
    optimizeSerialization(url);

    return exporter;
}
// DubboProtocol.java
private void openServer(URL url) {
    // find server.
    String key = url.getAddress();
    //client can export a service which's only for server to invoke
    // Only the service provider will start listening
    boolean isServer = url.getParameter(Constants.IS_SERVER_KEY, true);
    if (isServer) {
        ExchangeServer server = serverMap.get(key);
        if (server == null) {
            synchronized (this) {
                server = serverMap.get(key);
                if (server == null) {
                    // Create server instance
                    serverMap.put(key, createServer(url));
                }
            }
        } else {
            // server supports reset, use together with override
            server.reset(url);
        }
    }
}

Here is the part of creating a server and binding ports. Don't chase. Draw a flow chart. Chase it yourself. You won't faint.

// DubboProtocol.java
private ExchangeServer createServer(URL url) {

    // Omit some codes

    ExchangeServer server;
    try {
        // The default selection for transmission is
        server = Exchangers.bind(url, requestHandler);
    } catch (RemotingException e) {
        throw new RpcException("Fail to start server(url: " + url + ") " + e.getMessage(), e);
    }

    // Omit some codes

    return server;
}

Draw a picture to summarize the process of service startup

Because NettyServer is started by default in the end, and Netty processes business logic through ChannelHandler, let's take a look at which ChannelHandler is added to NettyServer, including codec handler, IdleStateHandler and NettyServerHandler.

Is all logic handled by NettyServerHandler? Of course not. In the later part of the request processing, we will continue to pursue this part.

final NettyServerHandler nettyServerHandler = new NettyServerHandler(getUrl(), this);
channels = nettyServerHandler.getChannels();

bootstrap.group(bossGroup, workerGroup)
        .channel(NioServerSocketChannel.class)
        .childOption(ChannelOption.TCP_NODELAY, Boolean.TRUE)
        .childOption(ChannelOption.SO_REUSEADDR, Boolean.TRUE)
        .childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
        .childHandler(new ChannelInitializer<NioSocketChannel>() {
            @Override
            protected void initChannel(NioSocketChannel ch) throws Exception {
                // FIXME: should we use getTimeout()?
                int idleTimeout = UrlUtils.getIdleTimeout(getUrl());
                NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
                ch.pipeline()//.addLast("logging",new LoggingHandler(LogLevel.INFO))//for debug
                        .addLast("decoder", adapter.getDecoder()) // Decoder handler
                        .addLast("encoder", adapter.getEncoder()) // Encoder handler
                        // Heartbeat check handler
                        .addLast("server-idle-handler", new IdleStateHandler(0, 0, idleTimeout, MILLISECONDS))
                        .addLast("handler", nettyServerHandler);
            }
        });

Welcome to pay attention

Reference blog

URL unified model in Dubbo
[1]https://dubbo.apache.org/zh-cn/blog/introduction-to-dubbo-url.html
dubbo token
[2]https://www.jianshu.com/p/1a97f62ae663
A good series of articles
[3]https://www.bookstack.cn/read/apache-dubbo-2.7-source_code_guide/5e7559bb72a4ec11.md
[4]https://blog.csdn.net/qq_35190492/article/details/108345229

Tags: Dubbo

Posted by nicx on Sun, 08 May 2022 02:14:02 +0300