Build your own MVC framework based on Koa2

Express and Koa are lightweight web frameworks. Although they are flexible and simple, and you can start the server in a few lines of code, with the complexity of the business, you will soon find that you need to manually configure various middleware. Moreover, because this kind of web framework does not restrict the directory structure of the project, the project quality built by programmers at different levels is also very different. In order to solve the above problems, various upper web frameworks based on express and Koa have emerged in the community, such as Egg.js and Nest.js

My current company has also implemented a set of MVC development framework based on Koa and combined with its own business needs. The Node of our company is mainly used to undertake the BFF layer and does not involve the real business logic. Therefore, the framework only encapsulates the Koa relatively simply, with some general business components built in (such as authentication, proxy forwarding), through the agreed directory structure, automatic injection routing and some global methods

Recently, I took a simple look at the source code of the framework during fishing time, and the harvest was still great, so I decided to implement a toy version of MVC framework

Source code address

Frame use

Reference code - step1

│  app.js
│  routes.js
│  
├─controllers
│   home.js
│      
├─middlewares
│   index.js
│      
├─my-node-mvc # The framework we will implement later
|
|
├─services
│   home.js
│      
└─views
    home.html       

My node MVC is the MVC framework we will implement later. First, let's see the final use effect

routes.js

const routes = [
  {
    match: '/',
    controller: 'home.index'
  },
  {
    match: '/list',
    controller: 'home.fetchList',
    method: 'post'
  }
];
module.exports = routes;

middlewares/index.js

const middleware = () => {
  return async (context, next) => {
    console.log('Custom Middleware');
    await next()
  }
}
module.exports = [middleware()];

app.js

const { App } = require('./my-node-mvc');
const routes = require('./routes');
const middlewares = require('./middlewares');

const app = new App({
  routes,
  middlewares,
});

app.listen(4445, () => {
  console.log('app start at: http://localhost:4445');
})

My node MVC exposes an App class. We pass in two parameters, routes and middlewares, to tell the framework how to render routes and start middleware

We visit http://localhost:4445 First, it will go through our custom middleware

async (context, next) => {
  console.log('Custom Middleware');
  await next()
}

Then it will match to routes This path in JS

{
  match: '/',
  controller: 'home.index'
}

Then the framework goes back to find Home under the controllers folder JS, create a new Home object and call its index method, so the page renders Home under the views directory folder html

controllers/home.js

const { Controller } = require('../my-node-mvc');

// A controller parent class is exposed, so all controllers inherit it before injecting this CTX object

// this.ctx not only has the methods and attributes of koa, but also has the custom methods and attributes extended by my node MVC framework
class Home extends Controller {
  async index() {
    await this.ctx.render('home');
  }

  async fetchList() {
    const data = await this.ctx.services.home.getList();
    ctx.body = data;
  }
}

module.exports = Home;

Similar access http://localhost:4445/list It matches

{
  match: '/list',
  controller: 'home.fetchList'
}

So the fetchList method of the home object is called, which in turn calls the getList method of the home object under the services directory, and finally returns json data

services/home.js

const { Service } = require('../my-node-mvc')

const posts = [{
  id: 1,
  title: 'Fate/Grand Order',
}, {
  id: 2,
  title: 'Azur Lane',
}];

// A service parent class is exposed, so all services inherit it before injecting this CTX object
class Home extends Service {
  async getList() {
    return posts
  }
}

module.exports = Home

So far, the simplest MVC web process has been run through

< font color = "orange" > before starting the tutorial, you'd better hope that you have the reading experience of Koa source code. You can refer to my previous article: Analysis of Koa source code</font>

Next, we will implement the my node MVC framework step by step

Basic framework

Reference code - step2

My node MVC is based on Koa, so we need to install Koa first

npm i koa

my-node-mvc/app.js

const Koa = require('koa');

class App extends Koa {
  constructor(options={}) {
    super();
  }
}

module.exports = App;

We just need to simply extend and inherit the parent class Koa

my-node-mvc/index.js

// Export App
const App = require('./app');

module.exports = {
  App,
}

Let's test it

# Enter step 2 directory
cd step2
node app.js

visit http://localhost:4445/ Discovery server started successfully

Thus, the simplest package has been completed

Built in middleware

Our my node MVC framework needs to build some basic middleware, such as KOA bodyparser, KOA router, KOA views, etc. only in this way can we avoid the trouble of repeatedly installing middleware every time we build a new project

The built-in middleware is generally divided into two types:

  • Built in basic middleware: such as KOA bodyparser, KOA router, metrics, performance monitoring and health check
  • Built in business middleware: the framework integrates the common functions of each department into business middleware in combination with business requirements, such as single sign on and file upload
npm i uuid koa-bodyparser ejs koa-views

Let's try to create a new business middleware

my-node-mvc/middlewares/init.js

const uuid = require('uuid');

module.exports = () => {
  // A requestId is generated for each request
  return async (context, next) => {
    const id = uuid.v4().replace(/-/g, '')
    context.state.global = {
      requestId: id
    }
    await next()
  }
}

my-node-mvc/middlewares/index.js

const init = require('./init');
const views = require('koa-views');
const bodyParser = require('koa-bodyparser');

// Export the business middleware init and the basic middleware koa body parser koa views
module.exports = {
  init,
  bodyParser,
  views,
}

Now, we need to call these middleware during App initialization

my-node-mvc/index.js

const Koa = require('koa');
const middlewares = require('./middlewares');

class App extends Koa {
  constructor(options={}) {
    super();

    const { projectRoot = process.cwd(), rootControllerPath, rootServicePath, rootViewPath } = options;
    this.rootControllerPath = rootControllerPath || path.join(projectRoot, 'controllers');
    this.rootServicePath = rootServicePath || path.join(projectRoot, 'services');
    this.rootViewPath = rootViewPath || path.join(projectRoot, 'views');

    this.initMiddlewares();
  }

  initMiddlewares() {
    // Use this Use registration Middleware
    this.use(middlewares.init());
    this.use(middlewares.views(this.rootViewPath, { map: { html: 'ejs' } }))
    this.use(middlewares.bodyParser());
  }
}

module.exports = App;

Start step 2 / APP after modification js

app.use((ctx) => {
  ctx.body = ctx.state.global.requestId
})

app.listen(4445, () => {
  console.log('app start at: http://localhost:4445');
})

So every visit http://localhost:4445 Can return different requestids

Business Middleware

In addition to the built-in middleware of my node MVC, we can also import the middleware written by ourselves and let my node MVC start it for us

step2/app.js

const { App } = require('./my-node-mvc');
const routes = require('./routes');
const middlewares = require('./middlewares');

// The middlewares passed into our business middleware is an array
const app = new App({
  routes,
  middlewares,
});

app.use((ctx, next) => {
  ctx.body = ctx.state.global.requestId
})

app.listen(4445, () => {
  console.log('app start at: http://localhost:4445');
})

my-node-mvc/index.js

const Koa = require('koa');
const middlewares = require('./middlewares');

class App extends Koa {
  constructor(options={}) {
    super();
    this.options = options;

    this.initMiddlewares();
  }

  initMiddlewares() {
    // Receive incoming business middleware
    const { middlewares: businessMiddlewares } = this.options;
    // Use this Use registration Middleware
    this.use(middlewares.init())
    this.use(middlewares.bodyParser());

    // Initialize business middleware
    businessMiddlewares.forEach(m => {
      if (typeof m === 'function') {
        this.use(m);
      } else {
        throw new Error('Middleware must be a function');
      }
    });
  }
}

module.exports = App;

So our business middleware can be started successfully

step2/middlewares/index.js

const middleware = () => {
  return async (context, next) => {
    console.log('Custom Middleware');
    await next()
  }
}

module.exports = [middleware()];

Global method

We know that many methods have been mounted on the built-in object CTX of Koa, such as CTX cookies. get() ctx. Remove () and so on. In our my node MVC framework, we can actually add some global methods

How to continue adding methods on ctx? The conventional idea is to write a middleware and mount the method on ctx:

const utils = () => {
  return async (context, next) => {
    context.sayHello = () => {
      console.log('hello');
    }
    await next()
  }
}

// Using middleware
app.use(utils());

// Later middleware can use this method
app.use((ctx, next) => {
  ctx.sayHello();
})

However, this requires us to put utils middleware at the top level, so that later middleware can continue to use this method

We can change our thinking: every time the client sends an http request, Koa will call the createContext method, which method A new ctx will be returned, and then the ctx will be passed to various middleware

The key point is createContext. We can rewrite the createContext method and inject our global method before passing ctx to the middleware

my-node-mvc/index.js

const Koa = require('koa');

class App extends Koa {
  
  createContext(req, res) {
    // Call parent method
    const context = super.createContext(req, res);
    // Injection global method
    this.injectUtil(context);

    // Return ctx
    return context
  }

  injectUtil(context) {
    context.sayHello = () => {
      console.log('hello');
    }
  }
}

module.exports = App;

Matching route

Reference code - step3

We specify the routing rules of the framework:

const routes = [
  {
    match: '/', // Matching path
    controller: 'home.index', // Matching controller and method
    middlewares: [middleware1, middleware2], // The routing level middleware first passes through the routing middleware and finally reaches a method of the controller
  },
  {
    match: '/list',
    controller: 'home.fetchList',
    method: 'post', // Match http request
  }
];

Think about how to realize this configuration routing through koa router?

# https://github.com/ZijianHe/koa-router/issues/527#issuecomment-651736656
# koa-router 9. Path to regexp has been upgraded in version X
# router. Get ('/ *', (CTX) = > {CTX. Body = 'OK'}) becomes this way: router get('(.*)', (ctx) => { ctx.body = 'ok' })
npm i koa-router

New built-in routing Middleware
my-node-mvc/middlewares/router.js

const Router = require('koa-router');
const koaCompose = require('koa-compose');

module.exports = (routerConfig) => {
  const router = new Router();

  // Todo matches the routerConfig routing configuration sent in

  return koaCompose([router.routes(), router.allowedMethods()])
}

Note that I finally used koaCompose to combine the two methods into one, because the original method of KOA router needs to call use twice to register the middleware successfully

const router = new Router();

router.get('/', (ctx, next) => {
  // ctx.router available
});

app
  .use(router.routes())
  .use(router.allowedMethods());

After using KoaCompose, we only need to call use once when registering

class App extends Koa {
  initMiddlewares() {
    const { routes } = this.options;
    
    // Register routing
    this.use(middlewares.route(routes));
  }
}

Now let's implement the specific route matching logic:

module.exports = (routerConfig) => {
  const router = new Router();

  if (routerConfig && routerConfig.length) {
    routerConfig.forEach((routerInfo) => {
      let { match, method = 'get', controller, middlewares } = routerInfo;
      let args = [match];

      if (method === '*') {
        method = 'all'
      }

      if ((middlewares && middlewares.length)) {
        args = args.concat(middlewares)
      };

      controller && args.push(async (context, next) => {
        // Todo found controller
        console.log('233333');
        await next();
      });


      if (router[method] && router[method].apply) {
        // The wonderful use of apply
        // router.get('/demo', fn1, fn2, fn3);
        router[method].apply(router, args)
      }
    })
  }

  return koaCompose([router.routes(), router.allowedMethods()])
}

A clever trick of this code is to use an args array to collect routing information

{
  match: '/neko',
  controller: 'home.index',
  middlewares: [middleware1, middleware2],
  method: 'get'
}

If you want to match this routing information with koa router, it should be written as follows:

// middleware1 and middleware2 are the routing level middleware we sent in
// Finally, the request is passed to home Index method
router.get('/neko', middleware1, middleware2, home.index);

Since the matching rules are generated dynamically, we can't write them as above, so we have this skill:

const method = 'get';

// Collect dynamic rules through arrays
const args = ['/neko', middleware1, middleware2, async (context, next) => {
  // Call the controller method
  await home.index(context, next);
}];

// Finally, use apply
router[method].apply(router, args)

Inject Controller

In the previous routing middleware, we still lack the most critical step: find the corresponding Controller object

controller && args.push(async (context, next) => {
  // Todo found controller
  await next();
});

We have previously agreed that the controllers folder of the project stores the controller object by default, so just traverse the folder and find the object named home JS, and then call the corresponding method of the controller

npm i glob

Create my node MVC / loader / controller js

const glob = require('glob');
const path = require('path');

const controllerMap = new Map(); // Cache file name and corresponding path
const controllerClass = new Map(); // Cache file name and corresponding require object

class ControllerLoader {
  constructor(controllerPath) {
    this.loadFiles(controllerPath).forEach(filepath => {
      const basename = path.basename(filepath);
      const extname = path.extname(filepath);
      const fileName = basename.substring(0, basename.indexOf(extname));

      if (controllerMap.get(fileName)) {
        throw new Error(`controller Under the folder ${fileName}File with the same name!`)
      } else {
        controllerMap.set(fileName, filepath);
      }
    })
  }

  loadFiles(target) {
    const files = glob.sync(`${target}/**/*.js`)
    return files
  }

  getClass(name) {
    if (controllerMap.get(name)) {
      if (!controllerClass.get(name)) {
        const c = require(controllerMap.get(name));
        // This file is require d only when a controller is used
        controllerClass.set(name, c);
      }
      return controllerClass.get(name);
    } else {
      throw new Error(`controller Not under folder ${name}file`)
    }
  }

}

module.exports = ControllerLoader

Because there may be many files in the controllers folder, we don't need to require all the files when the project starts. When a request needs to call home controller, we can dynamically load the requirement ('/ my app / Controllers / home'). For the same module ID, the module will be cached when the node is loaded for the first time. When it is loaded again, it will be obtained from the cache

Modify my node MVC / APP js

const ControllerLoader = require('./loader/controller');
const path = require('path');

class App extends Koa {
  constructor(options = {}) {
    super();
    this.options = options;

    const { projectRoot = process.cwd(), rootControllerPath } = options;
    // The default controllers directory. You can also specify other paths by configuring the rootControllerPath parameter
    this.rootControllerPath = rootControllerPath || path.join(projectRoot, 'controllers'); 
    this.initController();
    this.initMiddlewares();
  }

  initController() {
    this.controllerLoader = new ControllerLoader(this.rootControllerPath);
  }

  initMiddlewares() {
    // Pass the controller loader to the routing Middleware
    this.use(middlewares.route(routes, this.controllerLoader))
  }
}

module.exports = App;

my-node-mvc/middlewares/router.js

// Omit other codes

controller && args.push(async (context, next) => {
  // Find controller home index
  const arr = controller.split('.');
  if (arr && arr.length) {
    const controllerName = arr[0]; // home
    const controllerMethod = arr[1]; // index
    const controllerClass = loader.getClass(controllerName); // Get class through loader

    // Every time the controller requests a new one, because the context is new every time
    // Pass in context and next
    const controller = new controllerClass(context, next);
    if (controller && controller[controllerMethod]) {
      await controller[controllerMethod](context, next);
    }
  } else {
    await next();
  }
});

Create my node MVC / controller js

class Controller {
  constructor(ctx, next) {
    this.ctx = ctx;
    this.next = next;
  }
}

module.exports = Controller;

Our my node MVC will provide a Controller base class. All business controllers should inherit from it, so we can get this in the method CTX

my-node-mvc/index.js

const App = require('./app');
const Controller = require('./controller');

module.exports = {
  App,
  Controller, // Expose Controller
}
const { Controller } = require('my-node-mvc');

class Home extends Controller {
  async index() {
    await this.ctx.render('home');
  }
}

module.exports = Home;

Injection Services

const { Controller } = require('my-node-mvc');

class Home extends Controller {
  async fetchList() {
    const data = await this.ctx.services.home.getList();
    ctx.body = data;
  }
}

module.exports = Home;

this. A services object will be mounted on the CTX object, which contains all the service objects under the services folder of the project root directory

Create my node MVC / loader / service js

const path = require('path');
const glob = require('glob');

const serviceMap = new Map();
const serviceClass = new Map();
const services = {};

class ServiceLoader {
  constructor(servicePath) {
    this.loadFiles(servicePath).forEach(filepath => {
      const basename = path.basename(filepath);
      const extname = path.extname(filepath);
      const fileName = basename.substring(0, basename.indexOf(extname));

      if (serviceMap.get(fileName)) {
        throw new Error(`servies Under the folder ${fileName}File with the same name!`)
      } else {
        serviceMap.set(fileName, filepath);
      }

      const _this = this;

      Object.defineProperty(services, fileName, {
        get() {
          if (serviceMap.get(fileName)) {
            if (!serviceClass.get(fileName)) {
              // This file is require d only when a service is used
              const S = require(serviceMap.get(fileName));
              serviceClass.set(fileName, S);
            }
            const S = serviceClass.get(fileName);
            // Create a new Service instance each time
            // Incoming context
            return new S(_this.context);
          }
        }
      })

    });
  }

  loadFiles(target) {
    const files = glob.sync(`${target}/**/*.js`)
    return files
  }

  getServices(context) {
    // Update context
    this.context = context;
    return services;
  }

}

module.exports = ServiceLoader

Basic code of my-controller.mvc JS is a routine, just use object Defineproperty defines the get method of the services object, which calls services Home, you can automatically require ('/ my app / services / home')

Then, we also need to mount the services object to the ctx object. Remember how to define global methods before? Or the same routine (encapsulated thousand layer routine)

class App extends Koa {

  constructor() {
    this.rootViewPath = rootViewPath || path.join(projectRoot, 'views');
    this.initService();
  }

  initService() {
    this.serviceLoader = new ServiceLoader(this.rootServicePath);
  }

  createContext(req, res) {
    const context = super.createContext(req, res);
    // Injection global method
    this.injectUtil(context);

    // Injection Services
    this.injectService(context);

    return context
  }

  injectService(context) {
    const serviceLoader = this.serviceLoader;

    // Add services object to context
    Object.defineProperty(context, 'services', {
      get() {
        return serviceLoader.getServices(context)
      }
    })
  }
}

Similarly, we also need to provide a Service base class, and all business services should inherit from it

Create my node MVC / service js

class Service {
  constructor(ctx) {
    this.ctx = ctx;
  }
}

module.exports = Service;

my-node-mvc/index.js

const App = require('./app');
const Controller = require('./controller');
const Service = require('./service');

module.exports = {
  App,
  Controller,
  Service, // Expose Service
}
const { Service } = require('my-node-mvc');

const posts = [{
  id: 1,
  title: 'this is test1',
}, {
  id: 2,
  title: 'this is test2',
}];

class Home extends Service {
  async getList() {
    return posts;
  }
}

module.exports = Home;

summary

Based on Koa2, this paper encapsulates a very basic MVC framework from scratch. I hope to provide readers with some ideas and inspiration for framework encapsulation. For more framework details, please see what I wrote little-node-mvc

Of course, the encapsulation of this article is very simple. You can continue to improve more functions in combination with the actual situation of the company: for example, provide a my node MVC template project template, and develop a command-line tool my node MVC cli to pull and create the template

Among them, the combination of built-in middleware and framework can be regarded as injecting the real soul into the packaging. Our company has encapsulated many general business middleware: authentication, logging, performance monitoring, full link tracking, configuration center and other private NPM packages. It can be easily integrated through the self-developed Node framework. At the same time, we use scaffolding tools to provide out of the box project templates, It reduces many unnecessary development and operation and maintenance costs for the business

Tags: node.js Front-end koa2

Posted by andymt on Wed, 25 May 2022 18:10:42 +0300