对象存储 > 使用指南 > 开发指南 > AWS S3 兼容 > 兼容 SDK 示例 > AWS SDK for JavaScript

AWS SDK for JavaScript

最近更新时间: 2024-02-19 17:02:54

AWS SDK for JavaScript 可以同时支持 NodeJS 和浏览器。

从 NodeJS 导入 AWS SDK for JavaScript

确保 Node.js v13 或更高版本，并确保 npm，npx 已经安装。

初始化 TypeScript 项目

npm i typescript@5.2.2 ts-node@10.9.1 @types/node@20.8.4 --save-dev
npx tsc --init

导入 AWS SDK for JavaScript

npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0

对于之后的每个代码示例，将代码创建在 index.ts 后，执行

npx ts-node index.ts

即可执行代码。

从浏览器导入 AWS SDK for JavaScript

AWS SDK for JavaScript 支持的浏览器列表可以参见官方文档

初始化 TypeScript 项目

npm i webpack@5.89.0 webpack-cli@5.1.4 @webpack-cli/generators@3.0.7 path-browserify@1.0.1 --save-dev
npx webpack init

设置 tsconfig.json，确保以下选项被设置

{
  "compilerOptions": {
    "module": "NodeNext",
    "moduleResolution": "NodeNext"
  }
}

导入 AWS SDK for JavaScript

npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0

执行 Webpack

npm run build

启动 Webpack dev server

npx webpack serve

该命令会自动启动浏览器，导入 src/index.ts，执行代码。

需要注意：从浏览器访问 S3 接口可能需要修改跨域配置。另外以下部分代码实例因为使用了 NodeJS 库因此不适用于浏览器场景。

对象上传

获取客户端上传 URL

创建 index.ts

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }))
  .then((data) => {
    console.log(data);
  })
  .catch((err) => {
    console.error(err);
  });

这段代码将生成一个经过预先签名的客户端上传 URL，有效期为 900 秒，客户端可以在过期时间内对该 URL 发送 HTTP PUT 请求将文件上传。

以下是用 curl 上传文件的案例：

curl -X PUT --upload-file "<path/to/file>" "<presigned url>"

您也可以自行指定上传凭证的有效期，例如：

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), {
  expiresIn: 3600,
})
  .then((data) => {
    console.log(data);
  })
  .catch((err) => {
    console.error(err);
  });

服务器端直传

单请求上传（文件）

创建 index.ts

import * as fs from "fs";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

const fileStream = fs.createReadStream("<path/to/upload>");
fileStream.on("error", (err) => console.error(err));

s3.send(
  new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", Body: fileStream })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

单请求上传（数据流）

创建 index.ts

import { Readable } from "stream";
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new PutObjectCommand({
    Bucket: "<Bucket>",
    Key: "<Key>",
    Body: Readable.from("Hello, Qiniu S3!"),
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

分片上传（文件）

创建 index.ts

import * as fs from "fs";
import { StreamingBlobPayloadInputTypes } from "@smithy/types";
import {
  S3Client,
  CreateMultipartUploadCommand,
  CreateMultipartUploadCommandOutput,
  UploadPartCommand,
  UploadPartCommandOutput,
  CompletedPart,
  CompleteMultipartUploadCommand,
  CompleteMultipartUploadCommandOutput,
} from "@aws-sdk/client-s3";

async function createMultipartUpload(
  s3: S3Client,
  bucket: string,
  key: string
): Promise<CreateMultipartUploadCommandOutput> {
  return s3.send(
    new CreateMultipartUploadCommand({
      Bucket: bucket,
      Key: key,
    })
  );
}

async function uploadPart(
  s3: S3Client,
  bucket: string,
  key: string,
  uploadId: string,
  partNumber: number,
  body: StreamingBlobPayloadInputTypes,
  contentLength: number
): Promise<UploadPartCommandOutput> {
  return s3.send(
    new UploadPartCommand({
      Bucket: bucket,
      Key: key,
      UploadId: uploadId,
      PartNumber: partNumber,
      Body: body,
      ContentLength: contentLength,
    })
  );
}

async function completeMultipartUpload(
  s3: S3Client,
  bucket: string,
  key: string,
  uploadId: string,
  parts: CompletedPart[]
): Promise<CompleteMultipartUploadCommandOutput> {
  const cmd = new CompleteMultipartUploadCommand({
    Bucket: bucket,
    Key: key,
    UploadId: uploadId,
    MultipartUpload: {
      Parts: parts,
    },
  });
  return s3.send(cmd);
}

async function uploadParts(
  s3: S3Client,
  bucket: string,
  key: string,
  uploadId: string,
  filePath: string | Buffer | URL
): Promise<CompletedPart[]> {
  const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB
  const { size: fileSize } = await fs.promises.stat(filePath);
  const parts: CompletedPart[] = [];
  // 这里给出的案例是串行分片上传。可以自行改造成并行分片上传以进一步提升上传速度
  for (
    let offset = 0, partNum = 1;
    offset < fileSize;
    offset += PART_SIZE, partNum++
  ) {
    const options = {
      start: offset,
      end: Math.min(offset + PART_SIZE, fileSize) - 1,
    };
    const uploadPartCommandOutput = await uploadPart(
      s3,
      bucket,
      key,
      uploadId,
      partNum,
      fs.createReadStream(filePath, options),
      options.end + 1 - options.start
    );
    parts.push({ PartNumber: partNum, ETag: uploadPartCommandOutput.ETag });
  }
  return parts;
}

async function uploadFile(
  s3: S3Client,
  bucket: string,
  key: string,
  filePath: string | Buffer | URL
): Promise<CompleteMultipartUploadCommandOutput> {
  const createMultipartUploadCommandOutput = await createMultipartUpload(
    s3,
    bucket,
    key
  );
  const completedParts = await uploadParts(
    s3,
    bucket,
    key,
    createMultipartUploadCommandOutput.UploadId!,
    filePath
  );
  return await completeMultipartUpload(
    s3,
    bucket,
    key,
    createMultipartUploadCommandOutput.UploadId!,
    completedParts
  );
}

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

uploadFile(s3, "<Bucket>", "<Key>", "<path/to/upload>")
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

上传文件

创建 index.ts

import { Upload } from "@aws-sdk/lib-storage";
import { S3Client } from "@aws-sdk/client-s3";
import * as fs from "fs";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

const upload = new Upload({
  client: s3,
  params: {
    Bucket: "<Bucket>",
    Key: "<Key>",
    Body: fs.createReadStream("<path/to/upload>"),
  },
});

upload
  .done()
  .then((resp) => {
    if ((resp as CompleteMultipartUploadCommandOutput).ETag) {
      console.log("ETag:", (resp as CompleteMultipartUploadCommandOutput).ETag);
    } else {
      console.log("Aborted");
    }
  })
  .catch((err) => {
    console.error("Error:", err);
  });

对象下载

获取客户端下载 URL

创建 index.ts

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }))
  .then((data) => {
    console.log(data);
  })
  .catch((err) => {
    console.error(err);
  });

这段代码将生成一个经过预先签名的客户端下载 URL，有效期为 900 秒，客户端可以在过期时间内对该 URL 发送 HTTP GET 请求将文件下载。

以下是用 curl 下载文件的案例：

curl -o "<path/to/download>" "<presigned url>"

您也可以自行指定下载凭证的有效期，例如：

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), {
  expiresIn: 3600,
})
  .then((data) => {
    console.log(data);
  })
  .catch((err) => {
    console.error(err);
  });

服务器端直接下载

创建 index.ts

import * as fs from "fs";
import {
  S3Client,
  GetObjectCommand,
  GetObjectCommandOutput,
} from "@aws-sdk/client-s3";
import { Writable } from "stream";
import Readable from "stream";

async function getObject(
  s3: S3Client,
  bucket: string,
  key: string,
  writable: Writable
): Promise<GetObjectCommandOutput> {
  const getObjectCommandOutput = await s3.send(
    new GetObjectCommand({
      Bucket: bucket,
      Key: key,
    })
  );
  if (getObjectCommandOutput.Body) {
    (getObjectCommandOutput.Body as Readable).pipe(writable);
  }
  return getObjectCommandOutput;
}

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

getObject(s3, "<Bucket>", "<Key>", fs.createWriteStream("<path/to/download>"))
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

对象管理

获取对象信息

创建 index.ts

import { S3Client, HeadObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new HeadObjectCommand({
    Bucket: "<Bucket>",
    Key: "<Key>",
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

修改对象 MimeType

创建 index.ts

import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new CopyObjectCommand({
    Bucket: "<Bucket>",
    Key: "<Key>",
    CopySource: "/<Bucket>/<Key>",
    ContentType: "<NewContentType>",
    MetadataDirective: "REPLACE",
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

修改对象存储类型

创建 index.ts

import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new CopyObjectCommand({
    Bucket: "<Bucket>",
    Key: "<Key>",
    CopySource: "/<Bucket>/<Key>",
    StorageClass: "GLACIER",
    MetadataDirective: "REPLACE",
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

复制对象副本

创建 index.ts

import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new CopyObjectCommand({
    Bucket: "<ToBucket>",
    Key: "<ToKey>",
    CopySource: "/<FromBucket>/<ToBucket>",
    MetadataDirective: "COPY",
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

复制对象副本（大于 5GB）

创建 index.ts

import * as fs from "fs";
import {
  S3Client,
  CreateMultipartUploadCommand,
  CreateMultipartUploadCommandOutput,
  UploadPartCopyCommand,
  UploadPartCopyCommandOutput,
  CompletedPart,
  CompleteMultipartUploadCommand,
  CompleteMultipartUploadCommandOutput,
  HeadObjectCommand,
  HeadObjectCommandOutput,
} from "@aws-sdk/client-s3";

async function createMultipartUpload(
  s3: S3Client,
  bucket: string,
  key: string
): Promise<CreateMultipartUploadCommandOutput> {
  return s3.send(
    new CreateMultipartUploadCommand({
      Bucket: bucket,
      Key: key,
    })
  );
}

async function uploadPartCopy(
  s3: S3Client,
  fromBucket: string,
  fromKey: string,
  toBucket: string,
  toKey: string,
  uploadId: string,
  partNumber: number,
  from: number,
  end: number
): Promise<UploadPartCopyCommandOutput> {
  return s3.send(
    new UploadPartCopyCommand({
      Bucket: toBucket,
      Key: toKey,
      UploadId: uploadId,
      PartNumber: partNumber,
      CopySource: "/" + fromBucket + "/" + fromKey,
      CopySourceRange: "bytes=" + from + "-" + (end - 1),
    })
  );
}

async function completeMultipartUpload(
  s3: S3Client,
  bucket: string,
  key: string,
  uploadId: string,
  parts: CompletedPart[]
): Promise<CompleteMultipartUploadCommandOutput> {
  const cmd = new CompleteMultipartUploadCommand({
    Bucket: bucket,
    Key: key,
    UploadId: uploadId,
    MultipartUpload: {
      Parts: parts,
    },
  });
  return s3.send(cmd);
}

async function uploadPartsCopy(
  s3: S3Client,
  fromBucket: string,
  fromKey: string,
  toBucket: string,
  toKey: string,
  uploadId: string
): Promise<CompletedPart[]> {
  const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB
  const headObjectCommand = await s3.send(
    new HeadObjectCommand({
      Bucket: fromBucket,
      Key: fromKey,
    })
  );
  const contentLength: number = headObjectCommand.ContentLength!;
  const parts: CompletedPart[] = [];
  // 这里给出的案例是串行分片复制。可以自行改造成并行分片复制以进一步提升复制速度
  for (
    let offset = 0, partNum = 1;
    offset < contentLength;
    offset += PART_SIZE, partNum++
  ) {
    const uploadPartCommandOutput = await uploadPartCopy(
      s3,
      fromBucket,
      fromKey,
      toBucket,
      toKey,
      uploadId,
      partNum,
      offset,
      Math.min(offset + PART_SIZE, contentLength)
    );
    parts.push({
      PartNumber: partNum,
      ETag: uploadPartCommandOutput.CopyPartResult!.ETag,
    });
  }
  return parts;
}

async function copyFile(
  s3: S3Client,
  fromBucket: string,
  fromKey: string,
  toBucket: string,
  toKey: string
): Promise<CompleteMultipartUploadCommandOutput> {
  const createMultipartUploadCommandOutput = await createMultipartUpload(
    s3,
    toBucket,
    toKey
  );
  const completedParts = await uploadPartsCopy(
    s3,
    fromBucket,
    fromKey,
    toBucket,
    toKey,
    createMultipartUploadCommandOutput.UploadId!
  );
  return await completeMultipartUpload(
    s3,
    toBucket,
    toKey,
    createMultipartUploadCommandOutput.UploadId!,
    completedParts
  );
}

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

copyFile(s3, "<FromBucket>", "<FromKey>", "<ToBucket>", "<ToKey>")
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

删除空间中的文件

创建 index.ts

import { S3Client, DeleteObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new DeleteObjectCommand({
    Bucket: "<"Bucket>",
    Key: "<Key>",
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

获取指定前缀的文件列表

创建 index.ts

import { S3Client, ListObjectsCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new ListObjectsCommand({
    Bucket: "<Bucket>",
    Prefix: "<KeyPrefix>",
  })
)
  .then(({ Contents: contents }) => console.log(contents))
  .catch((err) => console.error(err));

批量删除空间中的文件

创建 index.ts

import { S3Client, DeleteObjectsCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "cn-east-1", // 华东-浙江区 region id
  endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: "<AccessKey>",
    secretAccessKey: "<SecretKey>",
  },
});

s3.send(
  new DeleteObjectsCommand({
    Bucket: "<Bucket>",
    Delete: {
      Objects: [
        {
          Key: "<Key1>",
        },
        {
          Key: "<Key2>",
        },
        {
          Key: "<Key3>",
        },
      ],
    },
  })
)
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

临时安全凭证

创建 index.ts

import { ListBucketsCommand, S3Client } from "@aws-sdk/client-s3";
import { STSClient, GetFederationTokenCommand } from "@aws-sdk/client-sts";

const sts = new STSClient({
  region: 'cn-east-1', // 华东-浙江区 region id
  endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint
  credentials: {
    accessKeyId: '<AccessKey>',
    secretAccessKey: '<SecretKey>',
  },
});

sts.send(new GetFederationTokenCommand({
  Name: 'Bob',
  DurationSeconds: 3600,
  Policy: '{"Version":"2012-10-17","Statement":[{"Sid":"Stmt1","Effect":"Allow","Action":["*"],"Resource":["*"]}]}',
}))
  .then((getFederationTokenOutput) => {
    const s3 = new S3Client({
      region: 'cn-east-1', // 华东-浙江区 region id
      endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint
      credentials: {
        accessKeyId: getFederationTokenOutput.Credentials!!.AccessKeyId!,
        secretAccessKey: getFederationTokenOutput.Credentials!!.SecretAccessKey!,
        sessionToken: getFederationTokenOutput.Credentials!!.SessionToken!,
        expiration: getFederationTokenOutput.Credentials!!.Expiration!,
      },
    });
    // 可以使用这些临时凭证调用 S3 服务
    s3.send(new ListBucketsCommand({}))
      .then((listBucketsOutput) => {
        for (const bucket of listBucketsOutput.Buckets!) {
            console.log(bucket.Name);
        }
      })
      .catch((err) => console.error(err));
  })
  .catch((err) => console.error(err));

以上内容是否对您有帮助？