Getting started

What is ElectricSheep.IO ?

ElectricSheep.IO is a (hopefully) simple tool to execute backup jobs over the network and copy archives offsite. It's designed to operate from a single utility machine over the network or the Internet using SSH. It eliminates the need to write scripts, configure Cron jobs on each target host.

🚧

Still under development

ElectricSheep.IO is still under development. It's working and has a good test coverage, but there is no stable release yet: configuration files, options and internal API are subject to change.

Installing it

Debian/Ubuntu

Download the package corresponding to your OS and system architecture. Install it using your package manager of choice or from the command line (as root):

dpkg -i electric-sheep-debian_0.5.0-1_amd64.deb

dpkg -i electric-sheep-debian_0.5.0-1_i386.deb

dpkg -i electric-sheep-ubuntu_0.5.0-1_amd64.deb

dpkg -i electric-sheep-ubuntu_0.5.0-1_i386.deb

📘

I can't find my distribution!

Right now installation packages are only available for Debian-based distribution, as .deb packages. Other distributions are coming next, feel free to let us know on which platform(s) you'd like to give it a try.

Alternative methods

A Docker image is available in the Docker Hub, see this cookbook.

System prerequisites

Although ElectricSheep.IO does not require a lot of computing resources, a worker (when run as a daemon) or standalone execution could temporarily eat memory up to a few hundreds of megabytes as they may have to manipulate large files in memory. The storage requirements depend on the size of your data and your backup policy.

Uninstalling

On Debian/Ubuntu:

dpkg -r electric-sheep

Upgrading

Uninstall the package and install the newest version.

How it works

Anatomy of a simple job

Each backup job is created in a configuration file called a Sheepfile. You can have as many configuration files you need and each Sheepfile can contain multiple jobs.

Each job typically refers to one or several hosts, and starts with a single resource such as a database, a file or a directory:

host "my-app-host", hostname: "host.example.com"

job "my-db-backup" do
  resource "database", name: "my-database", host: "my-app-host"
end

Starting with the original resource, a job can be viewed as a pipeline of commands and transports. Commands are usually executed on the remote hosts (or on the localhost) to transform the resource from the previous step, whereas transports allow us to move or copy the resource from a location to another. At last, notifiers allow you to send reports through various communication channels (here as an email):

host "my-app-host", hostname: "host.example.com"

job "my-db-backup" do
  resource "database", name: "my-database", host: "my-app-host"
  remotely as: "user" do
    # Create a MySQL dump
    mysql_dump user: "mysql-user", password: "secret"
    # Put the dump in an archive and delete the dump
    tar_gz delete_source: true
  end
  # Move the archive on the localhost
  move to: "localhost", using: "scp", as: "user"
  # Place an additional copy into the Cloud using Amazon S3
  copy to: "my-bucket/my-project", using: "s3", access_key_id: "XXX", secret_key: "secret"
  notify via: "email", using: "sendmail", from: "[email protected]", to: "[email protected]"
end

Configuring remote access

ES.IO relies on the SSH protocol to execute commands remotely (and to copy resources when using the SCP transport). By default, it uses the SSH key located at ~/.ssh/id_rsa. A good practice it to create and use a specific keypair so that you can easily add/remove it to/from remote hosts. To do so, create an SSH keypair and deploy the public key to the remote host(s):

ssh-keygen -t rsa -N '' -f ~/.ssh/electric_sheep_rsa
ssh-copy-id -i ~/.ssh/electric_sheep_rsa.pub [email protected]

You can specify the private key to use for authentication for each host:

host "my-app-host", hostname: "host.example.com", private_key: "~/.ssh/electric_sheep_rsa"

You can specify the specific ssh port to use for the connection for each host:

host "my-app-host", hostname: "host.example.com", ssh_port: 22222

🚧

Empty passphrase

So far, ElectricSheep.IO does not support agent-based authentication or passphrases.

Running ElectricSheep.IO

Standalone

Simply run ES.IO from the directory containing your Sheepfile:

electric_sheep

ES.IO will process backup jobs sequentially and exit.

As a daemon

Start a master process from the directory containing your Sheepfile:

electric_sheep start

The master process will look at jobs scheduling and fork to process the backup jobs when needed.

By default, the master will read and write the Sheepfile, PID file and log file in the current directory. See the Command Line Reference for startup options.

If you want to place the master process in the background and give control back to the terminal:

electric_sheep start -d