Decentralized Archive Transport
The Dat Protocol
Dat is a new p2p hypermedia protocol. It provides public-key- & sha256-addressed file archives which can be synced securely and browsed on-demand. Dat supports streaming updates and partial on-demand replication, and has plans for versioned URLs and efficient compaction.
|Fast||Archives sync from multiple sources at once.|
|Secure||Updates are signed, SHAed, and forced to be distributed uniformly.|
|Resilient||Archives can change hosts without changing their URLs.|
|Versioned||Changes are written to an append-only log.|
|Decentralized||Any device can host any archive.|
Dat is funded by grants by the Knight and Sloan Foundations. Learn more at the Dat Project.
» Join us in #dat on freenode. » File issues for this website here.
Draft, in progress. A complete specification of the Dat protocol.
How to create and download Dats using the shell.
Use the Dat CLI and a process-manager to rehost archives from a VPS.
Sync and share folders from the command-line.
A simple, GUI-based client for dat archives.
An experimental P2P browser, which uses Dat as an alternative to HTTP.
Dat uses rabin file-chunking and append-only feeds verified by merkle trees to securely distribute mutable files with "live" update-pushing through a peer-to-peer network. Peers are discovered using multiple overlapping networks (DHT, DNS, mUDP) and contacted via TCP or UTP.
Archives are identified by ed25519 public-keys or sha256 hashes encoded as 64-character hex strings, making the URLs hostless and globally unique.
The site you are currently viewing is served at
Dat generalizes concepts from certificate transparency to provide its security guarantees. All updates are written to a flattened merkle-tree (the feed) which is signed by the identifying public-key. Data-feeds are distributed uniformly to all peers, and authors are unable to split the change history or serve alternative versions of an archive without detection.
- Dat-node: A easy-to-use, comprehensive library for nodejs which wraps all modules.
- Hypercore: The internal protocol for distributing and replicating feeds of binary data.
- Hyperdrive: The internal protocol for reading and writing files using Hypercore.
- Hyperdiscovery: Join the p2p swarm for Dats.
There are many submodules which comprise Dat:
- Hyperdrive-encoding: The message encoding and schemas used by hyperdrive.
- Bitfield-rle: A run-length-encoder that compresses bitfields.
- Flat-tree: A series of functions to map a binary tree to a list.
- Protocol-buffers: Protocol Buffers for Node.js
- Uint64be: Encode / decode big endian unsigned 64 bit integers
- Discovery-Swarm: Uses the Mainline DHT, DNS Servers, and Multicast UDP to discover peers, and then establishes connections using TCP and UTP.
- Hypercore-protocol: A stream that implements the hypercore network protocol.
- Utp-native: Native bindings for libutp.
- Discovery-channel: Search for a key across multiple discovery networks and find peers who answer.
- Dns-discovery: Discover peers in a distributed system using regular dns and multicast dns.
- Bittorrent-dht: Discover peers with the BitTorrent DHT protocol.
There are also some handy community or high-level modules for handling common Dat tasks:
- Hyperdrive-http: Serve a Dat archive over HTTP.
- Hyperdrive-stats: Live & persistent stats module for Dat.
- Hyperdrive-network-speed: Track upload and download speed on a Dat archive.
- Hyperdrive-import-files: Import the contents of a folder into a Dat, and optionally keep watching for changes.
- Zip-to-hyperdrive: Import a .Zip into a Dat archive.
- Tar-dat: Import a .Tar into a Dat archive.
- Hyperdrive-to-zip-stream: Export a Dat archive into a Zip file.