adonisjs/drive

Filesystem provider built on top of node-flydrive

@adonisjs/drive

Introduction

A thin wrapper on top of Flydrive to work seamlessly with AdonisJS. The package comes pre-configured to serve files from the local filesystem using the AdonisJS HTTP server

Official Documentation

The documentation is available on the AdonisJS website

Contributing

One of the primary goals of AdonisJS is to have a vibrant community of users and contributors who believes in the principles of the framework.

We encourage you to read the contribution guide before contributing to the framework.

Code of Conduct

In order to ensure that the AdonisJS community is welcoming to all, please review and abide by the Code of Conduct.

License

AdonisJS Drive is open-sourced software licensed under the MIT license.

drive's Issues

Support AdonisJS v6

I read on https://v6-migration.adonisjs.com/guides/introduction#official-packages-not-yet-compatible-with-v6 that @adonisjs/drive is one of the official packages that yet has to be migrated to work with v6.

Is there any ETA for it? It keeps us from being able to finish the upgrade completely.

Bucket name (Question)

It's possible to define the bucket when move the file do disk using moveToDisk method?

Filename should stay consistent (Local vs S3)

Prerequisites

file.fileName after save or move to disk should keep a consistent name between all drive types. Uploading to a path within local will update the file name to filename.extension. Uploading to s3 will result in file name being path/filename.extension. Not really a bug just a proposed change.

Package version

"@adonisjs/core": "^5.4.0",
"@adonisjs/drive-s3": "^1.0.5"

Node.js and npm version

Node 16.4.0
Yarn 3.1.1

Sample Code

async function upload(file: MultipartFileContract) {
  await file.moveToDisk("path");
  console.log(file.fileName);
});

Change between s3 and local.

S3 file.fileName: 'path/filename.extension'
local file.fileName: 'filename.extension'

BONUS

Will provide a sample repo tomorrow if needed. Willing to do a PR, my idea is that it should either keep the path in the filename for local or strip the path out of filename when saving to S3 (since we already have a filePath on a file). Understandably s3 does not actually have file paths. Open to suggestions and not sure how this behaves on GCS.

Adonis S3 Driver is not working properly

Hello, im trying to upload images to s3 using the adonis s3 driver, but i am facing an issue with the contentType.

public async store({ request, params }: HttpContextContract) {
  const { file } = await request.validate(StoreValidator)

  const fileName = `${cuid()}.${file.extname}`
  const fileStream = fs.createReadStream(file.tmpPath!)

  await Drive.putStream(fileName, fileStream, {
    contentType: 'image/png' // or file.headers['content-type']
  })
}

As you could see above, im passing the contentType as "image/png", but it doesnt make any change on s3 actually, as you can see below:

add Url method to the local driver

while I'm using the s3 storage as my drive I can use method getUrl for return the media url but if I use the local storage I can not access to this method.

It should be a good Idea in the local drive we have property named url and it reference to the file base url and can use the getUrl in the local drive for return the file url

Cannot find module 'aws-sdk/clients/s3'

I have installed adonis-drive via:
adonis install @adonisjs/drive

My env file has these props:

S3_KEY="MY_S3_KEY"
S3_SECRET="MY_SECRET_KEY"
S3_BUCKET="MY_BUCKET"
S3_REGION="eu-central-1"

The config/drive.js file has been created automatically and I have set default: 's3' in there.

The problem:
When I run:
adonis repl
const Drive = use('Drive')
await Drive.exists('unicorn.jpg')

I get: Error: Cannot find module 'aws-sdk/clients/s3'

What have I done wrong here? Or is this a bug?

Thanks.

Adonis 5 Upgrade?

Is there any plan of upgrading this provider to Adonis v5?

I was looking at the code and I think I can be able to do it during my free time.

Can AdonisJS Drive be used without AdonisJS?

I'm using t3js (https://create.t3.gg/). Is it possible to use Drive without using Adonis? I checked that the dependencies don't list core.

Duplicate route name "drive.local.serve"

EDIT: This appears to be more an issue with Core, specifically the breaking of Drive into a separate package. Running Drive 2.0.7 with Core 5.2.3 works as expected, but in Core 5.2.4 the duplicate route issue appears.

---

After upgrading Drive to v2, Ace has begun throwing the error:

RouterException: E_DUPLICATE_ROUTE_NAME: Duplicate route name "drive.local.serve"

I've been following all the instructions at https://docs.adonisjs.com/guides/drive#configuration and the upgrade notes in https://docs.adonisjs.com/releases/august-2021-release#using-drive-in-existing-applications but no luck.

Package version

2.0.7 and 2.0.6. Interestingly 2.0.5 does not trigger any errors, but the "serveFiles" config variable name changed after this so might not be calling the affected code.

Node.js and npm version

Node: v15.12.0
NPM: 7.6.3

Sample Code (to reproduce the issue)

package.json

"dependencies": {
    "@adonisjs/auth": "^8.0.10",
    "@adonisjs/bouncer": "^2.2.5",
    "@adonisjs/core": "^5.4.0",
    "@adonisjs/drive": "^2.0.7",
    "@adonisjs/drive-s3": "^1.0.5",
    "@adonisjs/lucid": "^16.3.2",
    "@adonisjs/repl": "^3.1.7",
    ...

contracts/drive.ts

declare module '@ioc:Adonis/Core/Drive' {
  interface DisksList {
    local: {
      config: LocalDriverConfig
      implementation: LocalDriverContract
    }
  }
}

config/drive.ts

import Env from '@ioc:Adonis/Core/Env'
import { DriveConfig } from '@ioc:Adonis/Core/Drive'
import Application from '@ioc:Adonis/Core/Application'

const driveConfig: DriveConfig = {
  disk: Env.get('DRIVE_DISK'),

  disks: {
    local: {
      driver: 'local',
      visibility: 'public',
      root: Application.tmpPath('uploads'),
      serveFiles: true,
      basePath: '/uploads',
    },
  },
}

export default driveConfig

.env

DRIVE_DISK=local

MakeUrl is deprecated

Package version

2.0.9

Node.js and npm version

14.15.5 and 7.23.0

Sample Code (to reproduce the issue)

Use the method getUrl to print this message in the console

(node:30252) You are using legacy the API of the "Route.makeUrl". We recommend reading the docs and use the latest API: DeprecationWarning

This is created in this file and reading the doc, the new url builder must be used

[Config] local basePath doesn't support full URL like https://example.com/

Package version

2.3.0

Node.js and npm version

18.16.0, pnpm 8.2.0

Sample Code (to reproduce the issue)

I use local disk with basePath https://example.com/storage. Then I try to get URL:

const url = await Drive.getUrl('my-photo.png');

It returns:

/https://example.com/storage/my-photo.png

It should return:

https://example.com/storage/my-photo.png

BONUS (a sample repo to reproduce the issue)

Unable to store File

Node Version: v15.5.0
Adonis Version: 4.1

I'm trying to handle the process of streaming uploaded files to local and s3.

Error is thrown in the console:

(node:3892) [DEP0135] DeprecationWarning: WriteStream.prototype.open() is deprecated
(Use `node --trace-deprecation ...` to show where the warning was created)

never get back a response from the server loading forever.

here is my code snipped:

Route.post('/upload', async function ({request}) {
  const Drive = use('Drive');
  request.multipart.file('profile_pic', {}, async (file) => {
    console.log(file, "FILES")
    const path = await Drive.put(file.clientName, file.stream)
    console.log(path, "inside")
  })
  console.log( "outside")
  await request.multipart.process();
  return 'done';
});

the log writes in console FILES and outside. can't be reached inside log

BodyParser Config:

processManually: ['/upload']

array push is not working after Drive.putStream()

core version : "@adonisjs/core": "^5.8.6"

import { string } from '@IOC:Adonis/Core/Helpers'
import { createReadStream } from 'fs'
import { DateTime } from 'luxon'

let path : any[] = []
const fileStream = createReadStream(row.tmpPath!)
let fileName = string.camelCase(row.clientName).toLowerCase()
fileName = DateTime.now().toMillis() + "-" + fileName + "." + row.extname
await Drive.putStream(fileName, fileStream)

path.push({
path: fileName
})

Incorrect parameter definition on `fake` and `restore` functions

Package version

v2.3.0

Node.js and npm version

Node.js v16.13.1
npm v8.1.2

Sample Code (to reproduce the issue)

Drive.fake(['local', 'assets'])
//         ^^^^^^^^^^^^^^^^^^^
// TS2345: Argument of type 'string[]' is not assignable to parameter of type 'keyof DisksList | keyof DisksList[] | undefined'.

The expected type comes from the 'driver' property, which is declared here in the 'LocalDriverConfig & { driver: "local" type; }'

Package version

2.2.0

Node.js and npm version

Node.JS (16.13.1) | NPM (8.1.2)

Sample Code (to reproduce the issue)

I added S3 driver to the project with the command npm i @adonisjs/drive-s3 and then I did its configuration with node ace configure @adonisjs/drive-s3 but unfortunately the driver: 's3' is not found either generated again.

I don't know if it has any relationship, but this part of the code is missing the DriverContract and DriverConfig from s3 or another provider that I tested for the implementation of the drive

Configuration used.

BONUS (a sample repo to reproduce the issue)

Add New Driver

Hello, I'm trying to add Aliyun OSS support to Drive, but have no idea how to add a new driver using the extend method. Any chance someone could explain how the file structure should look like, how to register the new driver, etc? Thanks!

Typescript Error in drive.ts

I followed the instructions in https://docs.adonisjs.com/guides/drive#configuration but I got an error fromTypeScript.

config/drive.ts:

import Application from '@ioc:Adonis/Core/Application'
import { DriveConfig } from '@ioc:Adonis/Core/Drive'
import Env from '@ioc:Adonis/Core/Env'

const driveConfig: DriveConfig = {
  disk: Env.get('DRIVE_DISK'),  // 👈 Error: ts2322 type 'any' is not assignable to type 'never'

  disks: {
    local: {
      driver: 'local',
      visibility: 'public',
      root: Application.tmpPath('uploads'),
      basePath: '/uploads',
      serveAssets: true,
    },
  },
}

export default driveConfig

Add support for Uint8Array in addition to Buffer

It might just be a question of updating the TypeScript types. At least the Node.js fs APIs support both. I don't know about other drivers.

Return Drive.put() undefined.

I'm not able to get the return url when using Drive S3, it returns as undefined. I'm generating a PDF file and getting its buffer, but when trying it returns the url and returns as Promise { <pending> }.

Version:
"@adonisjs/drive-s3": "^1.3.1",

const redactions = await Database.from('redactions').select('*').orderBy('id', 'asc');

            const fonts = {
                Helvetica: {
                    normal: 'Helvetica',
                    bold: 'Helvetica-Bold',
                    italics: 'Helvetica-Oblique',
                    bolditalics: 'Helvetica-BoldOblique'
                },
            }

            const printer = new PDFPrinter(fonts)

            const body: any = []

            for await (let redaction of redactions) {
                const rows = new Array()
                rows.push(redaction.id)
                rows.push(redaction.title)
                rows.push(redaction.link)
                rows.push(redaction.not)
                body.push(rows)
            }

            const docDefinitions: TDocumentDefinitions = {
                defaultStyle: { font: "Helvetica" },
                content: [
                    {
                        table: {
                            body: [
                                ["id", "title", "link", "not"],
                                ...body,
                            ]
                        }
                    }
                ],
            }

            const pdfDoc = printer.createPdfKitDocument(docDefinitions)

            let chunks = [];
            var result;

            pdfDoc.on('data', (chunk: never) => {
                chunks.push(chunk);
            });
            pdfDoc.end();

            pdfDoc.on('end', async () => {
                result = Buffer.concat(chunks);
                console.log(result)
                const url = await Drive.put('teste.pdf', result, {
                    visibility: 'public',
                    contentType: 'application/pdf'
                })
                console.log(url)
            });

getStream returns old `NodeJS.ReadableStream` interface instead of Readable instance

Package version

"@adonisjs/drive-gcs": "^1.0.4",

Node.js and npm version

node v16.1.0
npm 7.11.2

Sample Code

Try to get a stream.

const stream = await Drive.getStream("bucket/fileName")
// stream is NodeJS.ReadableStream instead of the newer `Readable`

More info on compatibility section here: https://nodejs.org/api/stream.html#compatibility-with-older-nodejs-versions

Workaround

Wrapping the stream. See: https://nodejs.org/dist/latest-v16.x/docs/api/stream.html#readablewrapstream

Drive.getUrl is not raising exception when file is missing

Prerequisites

According to the documentation, Driver.getUrl should raise an error if the file is missing, but it seems broken.
Trying using local and s3 drivers, the behavior is that even if the file is missing, getUrl is returning an URL, which leads to a 404.

Package version

@adonisjs/core: 5.3.4
@adonisjs/driver-s3: 1.0.5

Node.js and npm version

node: 14.17.1
npm 6.14.13

Sample Code

Code is pretty straightforward.

Upload file:

await Drive.put(location, content)

Get URL file:

await Drive.getUrl(filename)

With local driver I'm getting: /uploads/b2786ed72a9faecbdd4c0d001163e095d7fd06d0b8b9bcf9b588e81863be1135

With s3 driver I'm getting: https://s3.eu-central-1.amazonaws.com/bucket-name-redacted/b2786ed72a9faecbdd4c0d001163e095d7fd06d0b8b9bcf9b588e81863be1135

Bonus

What I can add is that Driver.get is working as expected; in fact using this workaround I can "fix" the issue:

public async getUserProfilePicture(user: User): Promise<string | null> {
    const filename = user.encryptedEmail
    try {
        await Drive.get(filename)
        return Drive.getUrl(filename)
    } catch (e) {
        ctx.logger.info('Missing user profile picture')
        return null
    }
}

'Missing required key 'Key' in params' when attempting to stream files to S3.

I am trying to stream files to S3. I have disabled automatic file processing for the route I am using. I have configured s3 correctly (I know this since I am able to stream filis via adonis repl).

This is how my controller looks like: (exactly like the demo)

const Drive = use('Drive')

Route.post('upload', async ({ request }) => {

  request.multipart.file('profile_pic', {}, async (file) => {
    await Drive.disk('s3').put(file.fileName, file.stream)
  })

  await request.multipart.process()
})

But I am getting this error from the server: 'Missing required key 'Key' in params'

Am I doing something wrong?

Drive.delete returning undefined

The following code is returning undefined, it works, but the returning is not working
const test = await Drive.delete(filepath)

Asking

How can I using this feature while I used "@adonisjs/core": "~5.0.4-preview-rc". Because in Adonis documentation doesnt mention of how to invoke or configure from drive.

can't access local file by http-server while file not named latin alphabet

Prerequisites

can't access local file by http-server while file not named latin alphabet

Package version

^5.4.2

Node.js and npm version

v14.17.6

Sample Code (to reproduce the issue)

BONUS (a sample repo to reproduce the issue)

repo steps:

1. make a file named in Chinese like '你好.txt' in /tmp/uploads
2. access file by http://localhost:3333/uploads/你好.txt
3. then get file not found error

Finally, I found the reason is that @adonisjs/drive didn't parse file correct, look this picture

const location = request.param(LocalFileServer.filePathParamName).join('/');
// out put is ''%E4%BD%A0%E5%A5%BD.txt'

Then file driver can't find '%E4%BD%A0%E5%A5%BD.txt' because '你好.txt' was deal with encodeURIComponent.

I don't sure what the bug is @adonisjs/drive or @adonisjs/http-server or something, so I report here.

Thank for your reading.

Can this be used as a standalone package (without adonisjs)?

I came here while looking at flydrive.

Is it possible to use this package without adonisjs? Im in need of a storage abstraction library for another project but don't want (need) to integrate adonis into it (yet).