Aller au contenu principal
Version: 3.0.0

File upload

A general upload component for @midwayjs/faas, @midwayjs/web, @midwayjs/koa and @midwayjs/express frameworks, supporting two modes: file (temporary server file) and stream.

Related information:

Web support
@midwayjs/koa✅
@midwayjs/faas💬
@midwayjs/web✅
@midwayjs/express✅
attention

💬 Some function computing platforms do not support streaming request responses, please refer to the corresponding platform capabilities.

astuce

This module replaces the upload component since 3.17.0.

The differences from the upload component are:

    1. The configuration key is adjusted from upload to busboy
    1. The middleware is no longer loaded by default, and can be manually configured to the global or route
    1. the input parameter definition type is adjusted to UploadStreamFileInfo
    1. The configuration of fileSize has been adjusted

Install dependencies​

$ npm i @midwayjs/busboy@3 --save

Or add the following dependencies to package.json and reinstall.

{
"dependencies": {
  "@midwayjs/busboy": "^3.0.0",
    // ...
  },
  "devDependencies": {
    // ...
  }
}

Enable component​

// src/configuratin.ts

import { Configuration } from '@midwayjs/core';
import * as busboy from '@midwayjs/busboy';

@Configuration({
  imports: [
    // ...other components
    busboy
  ],
  // ...
})
export class MainConfiguration {}

Configure middleware​

The UploadMiddleware middleware is provided in the component, which can be configured globally or to a specific route. It is recommended to configure it to a specific route to improve performance.

Route Middleware

import { Controller, Post } from '@midwayjs/core';
import { UploadMiddleware } from '@midwayjs/busboy';

@Controller('/')
export class HomeController {

  @Post('/upload', { middleware: [UploadMiddleware] })
  async upload(/*...*/) {
    // ...
  }
}

Global Middleware

// src/configuratin.ts

import { Configuration } from '@midwayjs/core';
import * as busboy from '@midwayjs/busboy';
import { Application } from '@midwayjs/koa';

@Configuration({
  // ...
})
export class MainConfiguration {
  @App('koa')
  app: Application;
  
  async onReady() {
    this.app.useMiddleware(busboy.UploadMiddleware);
  }
}

Configuration​

The component uses busboy as the configuration key.

Upload mode​

Upload Modes​

There are three upload modes: file mode, stream mode, and the newly added async iterator mode.

In the code, the @Files() decorator is used to obtain the uploaded files, and the @Fields decorator is used to get other upload form fields.

file is the default value, with mode configured as the string file.

// src/config/config.default.ts
export default {
  // ...
  busboy: {
    mode: 'file',
  },
}

In the code, the uploaded files can be retrieved, and multiple files can be uploaded simultaneously.

import { Controller, Post, Files, Fields } from '@midwayjs/core';
import { UploadFileInfo } from '@midwayjs/busboy';

@Controller('/')
export class HomeController {

  @Post('/upload', /*...*/)
  async upload(@Files() files: Array<UploadFileInfo>, @Fields() fields: Record<string, string>) {
    /*
    files = [
      {
        filename: 'test.pdf',        // file name
        data: '/var/tmp/xxx.pdf',    // Server temporary file address
        mimeType: 'application/pdf', // mime
        fieldName: 'file'            // field name
      },
      // ...Support uploading multiple files at the same time under file
    ]
    */
  }
}

When using the file mode, the retrieved data represents the temporary file path of the uploaded file on the server. You can later handle the file contents using methods like fs.createReadStream. Multiple files can be uploaded at once, and they will be stored in an array.

Each object in the array contains the following fields:

export interface UploadFileInfo {
  /**
   * The name of the uploaded file
   */
  filename: string;
  /**
   * The MIME type of the uploaded file
   */
  mimeType: string;
  /**
   * The path where the file is saved on the server
   */
  data: string;
  /**
   * The form field name of the uploaded file
   */
  fieldName: string;
}

Upload file suffix check​

Use the whitelist property to configure the file suffixes allowed for upload. If null is configured, the suffix will not be checked.

attention

If null is configured, the uploaded file suffix will not be checked. If the file upload mode (mode=file) is adopted, it may be exploited by attackers to upload WebShells with suffixes such as .php and .asp to implement attack behaviors.

Of course, since the component will re-randomly generate the file name for the uploaded temporary file, as long as the developer does not return the uploaded temporary file address to the user, even if the user uploads some unexpected files, there is no need to worry too much about being exploited.

If the uploaded file suffix does not match, it will respond with 400 error. The default values are as follows:

'.jpg',
'.jpeg',
'.png',
'.gif',
'.bmp',
'.wbmp',
'.webp',
'.tif',
'.psd',
'.svg',
'.js',
'.jsx',
'.json',
'.css',
'.less',
'.html',
'.htm',
'.xml',
'.pdf',
'.zip',
'.gz',
'.tgz',
'.gzip',
'.mp3',
'.mp4',
'.avi',

The default suffix whitelist can be obtained through the uploadWhiteList exported in the component.

In addition, in order to prevent some malicious users from using certain technical means to forge some extensions that can be truncated, the midway upload component will filter the binary data of the obtained extensions, and only support characters in the range of 0x2e (that is, English dot .), 0x30-0x39 (that is, numbers 0-9), and 0x61-0x7a (that is, lowercase letters a-z) as extensions. Other characters will be automatically ignored.

You can pass a function to dynamically return the whitelist based on different conditions.

// src/config/config.default.ts
import { uploadWhiteList } from '@midwayjs/busboy';
import { tmpdir } from 'os';
import { join } from 'path';

export default {
  // ...
  busboy: {
    whitelist: (ctx) => {
      if (ctx.path === '/') {
        return [
          '.jpg',
          '.jpeg',
        ];
      } else {
        return [
          '.jpg',
        ]
      };
    },
    // ...
  },
}

Upload file MIME type check​

Some malicious users will try to modify the extension of WebShell such as .php to .jpg to bypass the whitelist filtering rules based on extensions. In some server environments, this jpg file will still be executed as a PHP script, causing security risks.

The component provides the mimeTypeWhiteList configuration parameter [Please note that this parameter has no default value, that is, it is not checked by default]. You can use this configuration to set the allowed file MIME format. The rule is a secondary array composed of the array [extension, mime, [...moreMime]], for example:

// src/config/config.default.ts
import { uploadWhiteList } from '@midwayjs/busboy';
export default {
  // ...
  busboy: {
    // ...
    // Extension whitelist
    whitelist: uploadWhiteList,
    // Only the following file types are allowed to be uploaded
    mimeTypeWhiteList: {
      '.jpg': 'image/jpeg',
      // You can also set multiple MIME types. For example, the following allows files with the .jpeg suffix to be either jpg or png.
      '.jpeg': ['image/jpeg', 'image/png'],
      // Other types
      '.gif': 'image/gif',
      '.bmp': 'image/bmp',
      '.wbmp': 'image/vnd.wap.wbmp',
      '.webp': 'image/webp',
    }
  },
}

You can also use the DefaultUploadFileMimeType variable provided by the component as the default MIME validation rule, which provides MIME data for commonly used file extensions such as .jpg, .png, and .psd:

// src/config/config.default.ts
import { uploadWhiteList, DefaultUploadFileMimeType } from '@midwayjs/busboy';
export default {
  // ...
  busboy: {
    // ...
    // Extension whitelist
    whitelist: uploadWhiteList,
    // Only the following file types are allowed to be uploaded
    mimeTypeWhiteList: DefaultUploadFileMimeType,
  },
}

You can query the file format and the corresponding MIME mapping through the website https://mimetype.io/. For the MIME identification of files, we use the npm package file-type@16. Please pay attention to the file types it supports.

info

MIME type verification rules only apply to file upload mode mode=file. After setting this verification rule, the upload performance will be slightly affected because the file content needs to be read for matching.

However, we still recommend that you set the mimeTypeWhiteList parameter when conditions permit, which will improve the security of your application.

You can pass a function that can dynamically return MIME rules based on different conditions.

// src/config/config.default.ts
import { tmpdir } from 'os';
import { join } from 'path';

export default {
  // ...
  busboy: {
    mimeTypeWhiteList: (ctx) => {
      if (ctx.path === '/') {
        return {
          '.jpg': 'image/jpeg',
        };
      } else {
        return {
          '.jpeg': ['image/jpeg', 'image/png'],
        }
      };
    }
  },
}

Busboy upload limit​

By default, there is no limit, which can be modified through configuration, digital type, unit is byte.

// src/config/config.default.ts
export default {
  // ...
  busboy: {
    // ...
    limits: {
      fileSize: 1024
    }
  },
}

In addition, you can set some other limits.

Temporary files and cleanup​

If you use the file mode to get the uploaded files, the uploaded files will be stored in the folder pointed to by the tmpdir option in the upload component configuration you set in the config file.

You can control the automatic temporary file cleanup time by using cleanTimeout in the configuration. The default value is 5 * 60 * 1000, that is, the uploaded files will be automatically cleaned up after 5 minutes. Setting it to 0 will disable the automatic cleanup function.

// src/config/config.default.ts
import { uploadWhiteList } from '@midwayjs/busboy';
import { tmpdir } from 'os';
import { join } from 'path';

export default {
  // ...
  busboy: {
    mode: 'file',
    tmpdir: join(tmpdir(), 'midway-busboy-files'),
    cleanTimeout: 5 * 60 * 1000,
  },
}

You can also call await ctx.cleanupRequestFiles() in the code to actively clean up temporary files uploaded by the current request.

Setting configurations for different routes​

Different middleware instances can be used to configure different routes differently. In this scenario, the global configuration will be merged and only a small part of the configuration can be covered.

import { Controller, Post, Files, Fields } from '@midwayjs/core';
import { UploadFileInfo, UploadMiddleware } from '@midwayjs/busboy';

@Controller('/')
export class HomeController {
  @Post('/upload1', { middleware: [ createMiddleware(UploadMiddleware, {mode: 'file'}) ]})
  async upload1(@Files() files Array<UploadFileInfo>) {
    // ...
  }
  
  @Post('/upload2', { middleware: [ createMiddleware(UploadMiddleware, {mode: 'stream'}) ]})
  async upload2(@Files() files Array<UploadFileInfo>) {
    // ...
  }
}

Currently the configurations that can be passed include mode and busboy's own configuration.

Built-in errors​

The following errors will be automatically triggered in different modes.

  • MultipartInvalidFilenameError Invalid file name
  • MultipartInvalidFileTypeError Invalid file type
  • MultipartFileSizeLimitError File size exceeds limit
  • MultipartFileLimitError Number of files exceeds limit
  • MultipartPartsLimitError Number of uploaded parts exceeds limit
  • MultipartFieldsLimitError Number of fields exceeds limit
  • MultipartError Other busbuy errors

Security tips​

  1. Please pay attention to whether to enable extension whitelist (whiteList). If the extension whitelist is set to null, it may be used by attackers to upload .php, .asp and other WebShells.

  2. Please pay attention to whether to set match or ignore rules, otherwise ordinary POST/PUT and other interfaces may be used by attackers, causing server load to increase and space to occupy a lot of problems.

  3. Please pay attention to whether file type rules (fileTypeWhiteList) are set, otherwise the attacker may forge the file type for uploading.

Front-end file upload example​

1. HTML form format​

<form action="/api/upload" method="post" enctype="multipart/form-data">
  Name: <input type="text" name="name" /><br />
  File: <input type="file" name="testFile" /><br />
  <input type="submit" value="Submit" />
</form>

2. fetch FormData method​

const fileInput = document.querySelector('#your-file-input') ;
const formData = new FormData();
formData.append('file', fileInput.files[0]);

fetch('/api/upload', {
  method: 'POST',
  body: formData,
});

Postman test example​