Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Created a File Class #97

Merged
merged 8 commits into from
May 29, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,5 +62,4 @@ typings/
# dotenv environment variables file
.env

index.d.ts
from.d.ts
*.d.ts
9 changes: 5 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
Changelog
=========

## v3.0.0-rc.0
## v3.0.0
- Changed WeakMap for private field (require node 12)
- Switch to ESM
- blob.stream() return a subset of whatwg stream which is the async iterable
- blob.stream() return a subset of whatwg stream which is the async iterable part
(it no longer return a node stream)
- Reduced the dependency of Buffer by changing to global TextEncoder/Decoder (require node 11)
- Disabled xo since it could understand private fields (#)
- No longer transform the type to lowercase (https://github.com/w3c/FileAPI/issues/43)
This is more loose than strict, keys should be lowercased, but values should not.
It would require a more proper mime type parser - so we just made it loose.
- index.js can now be imported by browser & deno since it no longer depends on any
core node features (but why would you? other environment can benefit from it)
- index.js and file.js can now be imported by browser & deno since it no longer depends on any
core node features (but why would you?)
- Implemented a File class

## v2.1.2
- Fixed a bug where `start` in BlobDataItem was undefined (#85)
Expand Down
34 changes: 22 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,11 @@ npm install fetch-blob
// Ways to import
// (PS it's dependency free ESM package so regular http-import from CDN works too)
import Blob from 'fetch-blob'
import File from 'fetch-blob/file.js'

import {Blob} from 'fetch-blob'
import {File} from 'fetch-blob/file.js'

const {Blob} = await import('fetch-blob')


Expand All @@ -105,27 +109,33 @@ globalThis.ReadableStream.from(blob.stream())
```

### Blob part backed up by filesystem
To use, install [domexception](https://github.com/jsdom/domexception).

```sh
npm install fetch-blob domexception
```
`fetch-blob/from.js` comes packed with tools to convert any filepath into either a Blob or a File
It will not read the content into memory. It will only stat the file for last modified date and file size.

```js
// The default export is sync and use fs.stat to retrieve size & last modified
// The default export is sync and use fs.stat to retrieve size & last modified as a blob
import blobFromSync from 'fetch-blob/from.js'
import {Blob, blobFrom, blobFromSync} from 'fetch-blob/from.js'
import {File, Blob, blobFrom, blobFromSync, fileFrom, fileFromSync} from 'fetch-blob/from.js'

const fsBlob1 = blobFromSync('./2-GiB-file.bin')
const fsBlob2 = await blobFrom('./2-GiB-file.bin')
const fsFile = fileFromSync('./2-GiB-file.bin', 'application/octet-stream')
const fsBlob = await blobFrom('./2-GiB-file.mp4')

// Not a 4 GiB memory snapshot, just holds 3 references
// Not a 4 GiB memory snapshot, just holds references
// points to where data is located on the disk
const blob = new Blob([fsBlob1, fsBlob2, 'memory'])
console.log(blob.size) // 4 GiB
const blob = new Blob([fsFile, fsBlob, 'memory', new Uint8Array(10)])
console.log(blob.size) // ~4 GiB
```

See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [tests](https://github.com/node-fetch/fetch-blob/blob/master/test.js) for more details.
`blobFrom|blobFromSync|fileFrom|fileFromSync(path, [mimetype])`

### Creating Blobs backed up by other async sources
Our Blob & File class are more generic then any other polyfills in the way that it can accept any blob look-a-like item
An example of this is that our blob implementation can be constructed with parts coming from [BlobDataItem](https://github.com/node-fetch/fetch-blob/blob/8ef89adad40d255a3bbd55cf38b88597c1cd5480/from.js#L32) (aka a filepath) or from [buffer.Blob](https://nodejs.org/api/buffer.html#buffer_new_buffer_blob_sources_options), It dose not have to implement all the methods - just enough that it can be read/understood by our Blob implementation. The minium requirements is that it has `Symbol.toStringTag`, `size`, `slice()` and either a `stream()` or a `arrayBuffer()` method. If you then wrap it in our Blob or File `new Blob([blobDataItem])` then you get all of the other methods that should be implemented in a blob or file

An example of this could be to create a file or blob like item coming from a remote HTTP request. Or from a DataBase

See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [tests](https://github.com/node-fetch/fetch-blob/blob/master/test.js) for more details of how to use the Blob.

[npm-image]: https://flat.badgen.net/npm/v/fetch-blob
[npm-url]: https://www.npmjs.com/package/fetch-blob
Expand Down
36 changes: 36 additions & 0 deletions file.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import Blob from './index.js';

export default class File extends Blob {
#lastModified = 0;
#name = '';

/**
* @param {*[]} fileBits
* @param {string} fileName
* @param {{lastModified?: number, type?: string}} options
*/ // @ts-ignore
constructor(fileBits, fileName, options = {}) {
if (arguments.length < 2) {
throw new TypeError(`Failed to construct 'File': 2 arguments required, but only ${arguments.length} present.`);
}
super(fileBits, options);

const modified = Number(options.lastModified);
this.#lastModified = Number.isNaN(this.#lastModified) ? modified : Date.now()
this.#name = fileName;
}

get name() {
return this.#name;
}

get lastModified() {
return this.#lastModified;
}

get [Symbol.toStringTag]() {
return "File";
}
}

export { File };
44 changes: 36 additions & 8 deletions from.js
Original file line number Diff line number Diff line change
@@ -1,26 +1,54 @@
import {statSync, createReadStream} from 'fs';
import {stat} from 'fs/promises';
import DOMException from 'domexception';
import {basename} from 'path';
import File from './file.js';
import Blob from './index.js';
import {MessageChannel} from 'worker_threads';

const DOMException = globalThis.DOMException || (() => {
const port = new MessageChannel().port1
const ab = new ArrayBuffer(0)
try { port.postMessage(ab, [ab, ab]) }
catch (err) { return err.constructor }
})()

/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const blobFromSync = (path, type) => fromBlob(statSync(path), path, type);
jimmywarting marked this conversation as resolved.
Show resolved Hide resolved

/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
jimmywarting marked this conversation as resolved.
Show resolved Hide resolved
*/
const blobFrom = (path, type) => stat(path).then(stat => fromBlob(stat, path, type));

/**
* @param {string} path filepath on the disk
* @returns {Blob}
* @param {string} [type] mimetype to use
*/
const blobFromSync = path => from(statSync(path), path);
const fileFrom = (path, type) => stat(path).then(stat => fromFile(stat, path, type));

/**
* @param {string} path filepath on the disk
* @returns {Promise<Blob>}
* @param {string} [type] mimetype to use
*/
const blobFrom = path => stat(path).then(stat => from(stat, path));
const fileFromSync = (path, type) => fromFile(statSync(path), path, type);

const fromBlob = (stat, path, type = '') => new Blob([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], {type});

const from = (stat, path) => new Blob([new BlobDataItem({
const fromFile = (stat, path, type = '') => new File([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})]);
})], basename(path), { type, lastModified: stat.mtimeMs });

/**
* This is a blob backed up by a file on the disk
Expand Down Expand Up @@ -72,4 +100,4 @@ class BlobDataItem {
}

export default blobFromSync;
export {Blob, blobFrom, blobFromSync};
export {File, Blob, blobFrom, blobFromSync, fileFrom, fileFromSync};
6 changes: 2 additions & 4 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ export default class Blob {
added += chunk.size
}
blobParts.push(chunk);
relativeStart = 0; // All next sequental parts should start at 0
relativeStart = 0; // All next sequential parts should start at 0

// don't add the overflow to new blobParts
if (added >= span) {
Expand All @@ -195,9 +195,7 @@ export default class Blob {

static [Symbol.hasInstance](object) {
return (
object &&
typeof object === 'object' &&
typeof object.constructor === 'function' &&
typeof object?.constructor === 'function' &&
(
typeof object.stream === 'function' ||
typeof object.arrayBuffer === 'function'
Expand Down
39 changes: 24 additions & 15 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
{
"name": "fetch-blob",
"version": "3.0.0-rc.0",
"description": "A Blob implementation in Node.js, originally from node-fetch.",
"version": "3.0.0",
"description": "Blob & File implementation in Node.js, originally from node-fetch.",
"main": "index.js",
"type": "module",
"files": [
"from.js",
"file.js",
"file.d.ts",
"index.js",
"index.d.ts",
"from.d.ts"
Expand All @@ -20,20 +22,23 @@
"repository": "https://github.com/node-fetch/fetch-blob.git",
"keywords": [
"blob",
"file",
"node-fetch"
],
"engines": {
"node": ">=14.0.0"
},
"author": "David Frank",
"author": "Jimmy Wärting <jimmy@warting.se> (https://jimmy.warting.se)",
"license": "MIT",
"bugs": {
"url": "https://github.com/node-fetch/fetch-blob/issues"
},
"homepage": "https://github.com/node-fetch/fetch-blob#readme",
"xo": {
"rules": {
"unicorn/import-index": "off",
"unicorn/prefer-node-protocol": "off",
"unicorn/numeric-separators-style": "off",
"unicorn/prefer-spread": "off",
"import/extensions": [
"error",
"always",
Expand All @@ -52,18 +57,22 @@
}
]
},
"peerDependenciesMeta": {
"domexception": {
"optional": true
}
},
"devDependencies": {
"ava": "^3.15.0",
"c8": "^7.7.1",
"codecov": "^3.8.1",
"domexception": "^2.0.1",
"c8": "^7.7.2",
"codecov": "^3.8.2",
"node-fetch": "^3.0.0-beta.9",
"typescript": "^4.2.4",
"xo": "^0.38.2"
}
"typescript": "^4.3.2",
"xo": "^0.40.1"
},
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/jimmywarting"
},
{
"type": "paypal",
"url": "https://paypal.me/jimmywarting"
}
]
}
74 changes: 71 additions & 3 deletions test.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import fs from 'fs';
import test from 'ava';
import {Response} from 'node-fetch';
import {Readable} from 'stream';
import buffer from 'buffer';
import test from 'ava';
import {Response} from 'node-fetch';
import syncBlob, {blobFromSync, blobFrom, fileFromSync, fileFrom} from './from.js';
import File from './file.js';
import Blob from './index.js';
import syncBlob, {blobFromSync, blobFrom} from './from.js';

const license = fs.readFileSync('./LICENSE', 'utf-8');

Expand Down Expand Up @@ -165,6 +166,21 @@ test('Reading after modified should fail', async t => {
// Change modified time
fs.utimesSync('./LICENSE', now, now);
const error = await blob.text().catch(error => error);
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
});

test('Reading file after modified should fail', async t => {
const file = fileFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 100);
});
const now = new Date();
// Change modified time
fs.utimesSync('./LICENSE', now, now);
const error = await file.text().catch(error => error);
jimmywarting marked this conversation as resolved.
Show resolved Hide resolved
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
});
Expand Down Expand Up @@ -239,6 +255,7 @@ test('Large chunks are divided into smaller chunks', async t => {
});

test('Can use named import - as well as default', async t => {
// eslint-disable-next-line node/no-unsupported-features/es-syntax
const {Blob, default: def} = await import('./index.js');
t.is(Blob, def);
});
Expand All @@ -254,3 +271,54 @@ if (buffer.Blob) {
t.is(await blob2.text(), 'blob part');
});
}

test('File is a instance of blob', t => {
t.true(new File([], '') instanceof Blob);
});

test('fileFrom returns the name', async t => {
t.is((await fileFrom('./LICENSE')).name, 'LICENSE');
});

test('fileFromSync returns the name', t => {
t.is(fileFromSync('./LICENSE').name, 'LICENSE');
});

test('fileFromSync(path, type) sets the type', t => {
t.is(fileFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});

test('blobFromSync(path, type) sets the type', t => {
t.is(blobFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});

test('fileFrom(path, type) sets the type', async t => {
const file = await fileFrom('./LICENSE', 'text/plain');
t.is(file.type, 'text/plain');
});

test('fileFrom(path, type) read/sets the lastModified ', async t => {
const file = await fileFrom('./LICENSE', 'text/plain');
// Earlier test updates the last modified date to now
t.is(typeof file.lastModified, 'number');
// The lastModifiedDate is deprecated and removed from spec
t.false('lastModifiedDate' in file);
t.is(file.lastModified > Date.now() - 60000, true);
});

test('blobFrom(path, type) sets the type', async t => {
const blob = await blobFrom('./LICENSE', 'text/plain');
t.is(blob.type, 'text/plain');
});

test('blobFrom(path) sets empty type', async t => {
const blob = await blobFrom('./LICENSE');
t.is(blob.type, '');
});

test('new File() throws with too few args', t => {
t.throws(() => new File(), {
instanceOf: TypeError,
message: 'Failed to construct \'File\': 2 arguments required, but only 0 present.'
});
});