Skip to content

Releases: storacha/ipfs-car

v1.2.0

22 Jan 11:53
1d6a8ad
Compare
Choose a tag to compare

1.2.0 (2024-01-22)

Features

v1.1.0

08 Dec 06:57
30b28d2
Compare
Choose a tag to compare

1.1.0 (2023-12-08)

Features

  • cli: check block hash consistency when listing blocks (#157) (273079f)

Bug Fixes

v1.0.0

20 Mar 15:46
31817e0
Compare
Choose a tag to compare

1.0.0 (2023-03-20)

⚠ BREAKING CHANGES

  • The programmatic API has changed significantly, see the README for new streaming API. The CLI "commands" like --pack have changed to pack (i.e. without dashes) but are largely very similar. In the CLI, CAR files written to stdout or piped to another program (i.e. not written to disk using --output) will not have a root CID in the CAR header. Minimum Node.js version for the CLI has changed to 18.

Features

CLI Migration Guide

Packing files into a CAR

Before:

ipfs-car --pack path/to/file/or/dir --output path/to/write/a.car

After:

ipfs-car pack path/to/file/or/dir --output path/to/write/a.car

Packing a file into a CAR, without wrapping in a directory

Before:

ipfs-car --pack path/to/files --wrapWithDirectory false --output path/to/write/a.car

After:

ipfs-car pack path/to/files --no-wrap --output path/to/write/a.car

Unpacking files

Before:

ipfs-car --unpack path/to/my.car --output /path/to/unpack/files/to

After:

ipfs-car unpack path/to/my.car --output /path/to/unpack/files/to

Listing root CIDs

Before:

ipfs-car --list-roots path/to/my.car

After:

ipfs-car roots path/to/my.car

Listing block CIDs

Before:

ipfs-car --list-cids path/to/my.car

After:

ipfs-car blocks path/to/my.car

Listing files

Before:

ipfs-car --list path/to/my.car

After:

ipfs-car ls path/to/my.car

Listing files with CIDs

Before:

ipfs-car --list-full path/to/my.car

After:

ipfs-car ls path/to/my.car --verbose

Generating CAR CID

Before:

ipfs-car --hash path/to/my.car

After:

ipfs-car hash path/to/my.car

API Migration Guide

Packing files into a CAR

The ipfs-car module now uses web streams. Note that due to the streaming nature of DAG and CAR generation the programmatic API does not produce CAR files with any roots in the CAR header.

import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

const file = new Blob(['Hello ipfs-car!'])
const carStream = createFileEncoderStream(file).pipeThrough(new CAREncoderStream())

// carStream.pipeTo(somewhereWritable)

Obtaining the root CID

The root CID is the CID of final block generated by the file/directory encoder stream. Use a transform stream to record the CID of the last block generated:

import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'

const file = new Blob(['Hello ipfs-car!'])
let rootCID

await createFileEncoderStream(file)
  .pipeThrough(new TransformStream({
    transform (block, controller) {
      rootCID = block.cid
      controller.enqueue(block)
    }
  }))
  .pipeThrough(new CAREncoderStream())
  .pipeTo(new WritableStream())

console.log(rootCID.toString())

Adding root CIDs to the CAR header

If you need root CIDs in the CAR header, there are two approaches you can use:

  1. Buffer all the DAG blocks, then encode with known root:

    import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'
    
    const file = new Blob(['Hello ipfs-car!'])
    const blocks = []
    
    // buffer the output
    await createFileEncoderStream(file)
      .pipeTo(new WritableStream({ write: b => blocks.push(b) }))
    
    const rootCID = blocks.at(-1).cid
    const blockStream = new ReadableStream({
      pull (controller) {
        if (blocks.length) {
          controller.enqueue(blocks.shift())
        } else {
          controller.close()
        }
      }
    })
    
    await blockStream
      .pipeThrough(new CAREncoderStream([rootCID])) // pass root to CAR encoder
      .pipeTo(new WritableStream())
  2. Write to disk with placeholder CID, then update after DAG is completely generated (Note: Node.js only):

    import fs from 'fs'
    import { Writable } from 'stream'
    import { CarWriter } from '@ipld/car/writer'
    import { CID } from 'multiformats/cid'
    import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'
    
    // Root CID written in CAR file header before it is updated with the real root CID.
    const placeholderCID = CID.parse('bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi')
    
    const file = new Blob(['Hello ipfs-car!'])
    let rootCID
    
    await createFileEncoderStream(file)
      .pipeThrough(new TransformStream({
        transform (block, controller) {
          rootCID = block.cid
          controller.enqueue(block)
        }
      }))
      .pipeThrough(new CAREncoderStream(placeholderCID))
      .pipeTo(Writable.toWeb(fs.createWriteStream('path/to/my.car')))
    
    // update roots in CAR header
    const fd = await fs.promises.open(opts.output, 'r+')
    await CarWriter.updateRootsInFile(fd, [rootCID])
    await fd.close()
Unpacking files from a CAR

This is no longer provided by this library, but is easy to do with @ipld/car and ipfs-unixfs-exporter modules:

import { CarIndexedReader } from '@ipld/car/indexed-reader'
import { recursive as exporter } from 'ipfs-unixfs-exporter'

const reader = await CarIndexedReader.fromFile('path/to/my.car')
const roots = await reader.getRoots()

const entries = exporter(roots[0], {
  async get (cid) {
    const block = await reader.get(cid)
    return block.bytes
  }
})

for await (const entry of entries) {
  if (entry.type === 'file' || entry.type === 'raw') {
    console.log('file', entry.path, entry.content)
  } else if (entry.type === 'directory') {
    console.log('directory', entry.path)
  }
}

v0.9.2

15 Feb 15:37
fdc46f5
Compare
Choose a tag to compare

0.9.2 (2023-02-15)

Bug Fixes

v0.9.1

03 Oct 15:45
679b6c0
Compare
Choose a tag to compare

0.9.1 (2022-10-03)

Bug Fixes

  • add car hash command to docs (baf23a6)

v0.9.0

03 Oct 15:33
56d328a
Compare
Choose a tag to compare

0.9.0 (2022-10-03)

Features

v0.8.1

05 Aug 11:12
4143da0
Compare
Choose a tag to compare

0.8.1 (2022-08-05)

Bug Fixes

  • readme contributing and license (e4cf245)

v0.8.0

02 Aug 10:11
cdc9617
Compare
Choose a tag to compare

0.8.0 (2022-08-02)

Features

v0.7.0

05 Apr 13:59
1ee839c
Compare
Choose a tag to compare

We now have an official IANA media type for CARs application/vnd.ipld.car so use it creating Blob

v0.6.2

08 Feb 16:29
853989a
Compare
Choose a tag to compare
chore: update @ipld/car and multiformats deps (#116)