« home

Guide: Hosting Archives from a Server

This guide will show you how to use some simple command-line tools to host archives on a VPS. (This guide is designed for systemd-based systems, including debian/ubuntu.)

TL;DR

Do this:


      npm install -g dat lil-pids add-to-systemd
      mkdir ~/dats
      echo "dat dat://ff34725120b2f3c5bd5028e4f61d14a45a22af48a7b12126d5d588becde88a93/ \
        ~/dats/datprotocol \
        --quiet" > ~/dats/services
      sudo add-to-systemd dat-lil-pids $(which lil-pids) ~/dats/services ~/dats/pids
      sudo systemctl start dat-lil-pids
      

Add and remove dat clone commands to ~/dats/services to change the archives your server hosts. You don't need to restart the dat-lil-pids service, because it watches the services file and automatically adds/removes processes to match it.

Step-by-step

Installation

On your server, install the following tools from npm:

npm install -g dat lil-pids add-to-systemd

Create the Archives

Follow the Dat CLI Guide to create the archives. You can do this on your local machine, or on the server. Take note of the URLs of the archives you want to rehost.

For this guide, we'll use this site (datprotocol.com) as an example archive. It is located at dat://ff34725120b2f3c5bd5028e4f61d14a45a22af48a7b12126d5d588becde88a93/.

Upload the Archives

To make sure the archives get to the server, we'll manually upload the archives first. In this guide, we'll use the ~/dats directory.


      mkdir ~/dats
      cd ~/dats
      dat clone dat://ff34725120b2f3c5bd5028e4f61d14a45a22af48a7b12126d5d588becde88a93/ ./datprotocol

      Created new dat in ~/dats/datprotocol/.dat
      Cloning: 131 files (2.5 GB)

      1 connection | Download 70 MB/s Upload 0 B/s

      Downloading updates...
      [=====-------------------------------------] 13.81%

      Ctrl+C to Exit
      

Once this is done, you can close the process.

Write the Services File

Now, we want to turn that 'clone' command into a long-running process. The lil-pids tool is a simple process-manager which watches a "services file" to determine which commands should be active. We'll use it to spawn a number of dat-clone processes. The file we'll write is ~/dats/services.

The command is the same as before, except that we'll add --upload to rehost the archive to the network, and --quiet to suppress output that we don't need cluttering our logs.

Use an editor to write to ~/dats/services:


      dat dat://ff34725120b2f3c5bd5028e4f61d14a45a22af48a7b12126d5d588becde88a93/ \
        ~/dats/datprotocol \
        --quiet
      

To start the commands, we run:


      lil-pids ~/dats/services ~/dats/pids

You can view the process ids of the commands (in another CLI session) with cat ~/dats/pids.

Add to Systemd

Finally, we want to turn the lil-pids process into a daemon. Run these commands to do so:


      sudo add-to-systemd dat-lil-pids $(which lil-pids) ~/dats/services ~/dats/pids
      sudo systemctl start dat-lil-pids
      

If successful, you'll be able to run sudo systemctl status dat-lil-pids and see the output.

If you want to add or remove dats, you don't need to restart the service. Just edit the ~/dats/services file to include the additional dat clone commands. After saving, lil-pids will automatically open and close processes to match the file.