{"_id":"54fda39021538c21006c49b6","createdAt":"2014-11-08T14:11:40.870Z","project":"545e249c7ca5470800b3a1b2","title":"Getting started","updates":[],"user":"545e246f7ca5470800b3a1b1","api":{"auth":"never","params":[],"results":{"codes":[]},"settings":"","url":""},"hidden":false,"link_external":false,"order":0,"sync_unique":"","body":"[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"What is ElectricSheep.IO ?\"\n}\n[/block]\n[ElectricSheep.IO](http://electricsheep.io) is a (hopefully) simple tool to execute backup jobs over the network and copy archives offsite. It's designed to operate from a single utility machine over the network or the Internet using SSH. It eliminates the need to write scripts, configure Cron jobs on each target host.\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"Still under development\",\n  \"body\": \"ElectricSheep.IO is still under development. It's working and has a good test coverage, but there is no stable release yet: configuration files, options and internal API are subject to change.\"\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Installing it\"\n}\n[/block]\n## Debian/Ubuntu\n\n[Download the package](/page/downloads) corresponding to your OS and system architecture. Install it using your package manager of choice or from the command line (as `root`):\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"dpkg -i electric-sheep-debian_0.5.0-1_amd64.deb\",\n      \"language\": \"shell\",\n      \"name\": \"Debian 64 bits\"\n    },\n    {\n      \"code\": \"dpkg -i electric-sheep-debian_0.5.0-1_i386.deb\",\n      \"language\": \"shell\",\n      \"name\": \"Debian 32 bits\"\n    },\n    {\n      \"code\": \"dpkg -i electric-sheep-ubuntu_0.5.0-1_amd64.deb\",\n      \"language\": \"shell\",\n      \"name\": \"Ubuntu 64 bits\"\n    },\n    {\n      \"code\": \"dpkg -i electric-sheep-ubuntu_0.5.0-1_i386.deb\",\n      \"language\": \"shell\",\n      \"name\": \"Ubuntu 32 bits\"\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"I can't find my distribution!\",\n  \"body\": \"Right now installation packages are only available for Debian-based distribution, as `.deb` packages. Other distributions are coming next, feel free to let us know on which platform(s) you'd like to give it a try.\"\n}\n[/block]\n## Alternative methods\n\nA Docker image is available in the Docker Hub, see this [cookbook](doc:docker).\n\n## System prerequisites\n\nAlthough ElectricSheep.IO does not require a lot of computing resources, a worker (when run as a daemon) or standalone execution could temporarily eat memory up to a few hundreds of megabytes as they may have to manipulate large files in memory. The storage requirements depend on the size of your data and your backup policy.\n\n## Uninstalling\n\nOn Debian/Ubuntu:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"dpkg -r electric-sheep\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Upgrading\n\nUninstall the package and install the newest version.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"How it works\"\n}\n[/block]\n## Anatomy of a simple job\n\nEach backup job is created in a configuration file called a _Sheepfile_. You can have as many configuration files you need and each _Sheepfile_ can contain multiple jobs.\n\nEach job typically refers to one or several hosts, and starts with a single resource such as a database, a file or a directory:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"host \\\"my-app-host\\\", hostname: \\\"host.example.com\\\"\\n\\njob \\\"my-db-backup\\\" do\\n  resource \\\"database\\\", name: \\\"my-database\\\", host: \\\"my-app-host\\\"\\nend\\n\",\n      \"language\": \"ruby\",\n      \"name\": \"Database - Sheepfile\"\n    }\n  ]\n}\n[/block]\nStarting with the original resource, a job can be viewed as a pipeline of commands and transports. Commands are usually executed on the remote hosts (or on the localhost) to transform the resource from the previous step, whereas transports allow us to move or copy the resource from a location to another. At last, notifiers allow you to send reports through various communication channels (here as an [email](doc:email)):\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"host \\\"my-app-host\\\", hostname: \\\"host.example.com\\\"\\n\\njob \\\"my-db-backup\\\" do\\n  resource \\\"database\\\", name: \\\"my-database\\\", host: \\\"my-app-host\\\"\\n  remotely as: \\\"user\\\" do\\n    # Create a MySQL dump\\n    mysql_dump user: \\\"mysql-user\\\", password: \\\"secret\\\"\\n    # Put the dump in an archive and delete the dump\\n    tar_gz delete_source: true\\n  end\\n  # Move the archive on the localhost\\n  move to: \\\"localhost\\\", using: \\\"scp\\\", as: \\\"user\\\"\\n  # Place an additional copy into the Cloud using Amazon S3\\n  copy to: \\\"my-bucket/my-project\\\", using: \\\"s3\\\", access_key_id: \\\"XXX\\\", secret_key: \\\"secret\\\"\\n  notify via: \\\"email\\\", using: \\\"sendmail\\\", from: \\\"robots:::at:::example.com\\\", to: \\\"me@example.com\\\"\\nend\",\n      \"language\": \"ruby\"\n    }\n  ]\n}\n[/block]\n## Configuring remote access\n\nES.IO relies on the SSH protocol to execute commands remotely (and to copy resources when using the SCP transport). By default, it uses the SSH key located at `~/.ssh/id_rsa`. A good practice it to create and use a specific keypair so that you can easily add/remove it to/from remote hosts. To do so, create an SSH keypair and deploy the public key to the remote host(s):\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ssh-keygen -t rsa -N '' -f ~/.ssh/electric_sheep_rsa\\nssh-copy-id -i ~/.ssh/electric_sheep_rsa.pub user@host.example.com\",\n      \"language\": \"shell\",\n      \"name\": \"SSH Setup\"\n    }\n  ]\n}\n[/block]\nYou can specify the private key to use for authentication for each host:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"host \\\"my-app-host\\\", hostname: \\\"host.example.com\\\", private_key: \\\"~/.ssh/electric_sheep_rsa\\\"\",\n      \"language\": \"ruby\",\n      \"name\": \"Sheepfile\"\n    }\n  ]\n}\n[/block]\nYou can specify the specific ssh port to use for the connection for each host:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"host \\\"my-app-host\\\", hostname: \\\"host.example.com\\\", ssh_port: 22222\",\n      \"language\": \"ruby\",\n      \"name\": \"Sheepfile\"\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"Empty passphrase\",\n  \"body\": \"So far, ElectricSheep.IO does not support agent-based authentication or passphrases.\"\n}\n[/block]\n## Running ElectricSheep.IO\n\n### Standalone\n\nSimply run ES.IO from the directory containing your _Sheepfile_:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"electric_sheep\",\n      \"language\": \"shell\",\n      \"name\": \"Run\"\n    }\n  ]\n}\n[/block]\nES.IO will process backup jobs sequentially and exit.\n\n### As a daemon\n\nStart a master process from the directory containing your _Sheepfile_:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"electric_sheep start\",\n      \"language\": \"shell\"\n    }\n  ]\n}\n[/block]\nThe master process will look at jobs scheduling and fork to process the backup jobs when needed.\n\nBy default, the master will read and write the _Sheepfile_, PID file and log file in the current directory. See the [Command Line Reference](doc:command-line) for startup options.\n\nIf you want to place the master process in the background and give control back to the terminal:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"electric_sheep start -d\",\n      \"language\": \"shell\"\n    }\n  ]\n}\n[/block]","category":"54fda38f21538c21006c49a7","excerpt":"This page will help you get started with ElectricSheep.IO","link_url":"","slug":"getting-started","type":"basic","version":"54fda38e21538c21006c49a6","__v":3,"githubsync":"","isReference":false,"childrenPages":[]}

Getting started

This page will help you get started with ElectricSheep.IO

[block:api-header] { "type": "basic", "title": "What is ElectricSheep.IO ?" } [/block] [ElectricSheep.IO](http://electricsheep.io) is a (hopefully) simple tool to execute backup jobs over the network and copy archives offsite. It's designed to operate from a single utility machine over the network or the Internet using SSH. It eliminates the need to write scripts, configure Cron jobs on each target host. [block:callout] { "type": "warning", "title": "Still under development", "body": "ElectricSheep.IO is still under development. It's working and has a good test coverage, but there is no stable release yet: configuration files, options and internal API are subject to change." } [/block] [block:api-header] { "type": "basic", "title": "Installing it" } [/block] ## Debian/Ubuntu [Download the package](/page/downloads) corresponding to your OS and system architecture. Install it using your package manager of choice or from the command line (as `root`): [block:code] { "codes": [ { "code": "dpkg -i electric-sheep-debian_0.5.0-1_amd64.deb", "language": "shell", "name": "Debian 64 bits" }, { "code": "dpkg -i electric-sheep-debian_0.5.0-1_i386.deb", "language": "shell", "name": "Debian 32 bits" }, { "code": "dpkg -i electric-sheep-ubuntu_0.5.0-1_amd64.deb", "language": "shell", "name": "Ubuntu 64 bits" }, { "code": "dpkg -i electric-sheep-ubuntu_0.5.0-1_i386.deb", "language": "shell", "name": "Ubuntu 32 bits" } ] } [/block] [block:callout] { "type": "info", "title": "I can't find my distribution!", "body": "Right now installation packages are only available for Debian-based distribution, as `.deb` packages. Other distributions are coming next, feel free to let us know on which platform(s) you'd like to give it a try." } [/block] ## Alternative methods A Docker image is available in the Docker Hub, see this [cookbook](doc:docker). ## System prerequisites Although ElectricSheep.IO does not require a lot of computing resources, a worker (when run as a daemon) or standalone execution could temporarily eat memory up to a few hundreds of megabytes as they may have to manipulate large files in memory. The storage requirements depend on the size of your data and your backup policy. ## Uninstalling On Debian/Ubuntu: [block:code] { "codes": [ { "code": "dpkg -r electric-sheep", "language": "text" } ] } [/block] ## Upgrading Uninstall the package and install the newest version. [block:api-header] { "type": "basic", "title": "How it works" } [/block] ## Anatomy of a simple job Each backup job is created in a configuration file called a _Sheepfile_. You can have as many configuration files you need and each _Sheepfile_ can contain multiple jobs. Each job typically refers to one or several hosts, and starts with a single resource such as a database, a file or a directory: [block:code] { "codes": [ { "code": "host \"my-app-host\", hostname: \"host.example.com\"\n\njob \"my-db-backup\" do\n resource \"database\", name: \"my-database\", host: \"my-app-host\"\nend\n", "language": "ruby", "name": "Database - Sheepfile" } ] } [/block] Starting with the original resource, a job can be viewed as a pipeline of commands and transports. Commands are usually executed on the remote hosts (or on the localhost) to transform the resource from the previous step, whereas transports allow us to move or copy the resource from a location to another. At last, notifiers allow you to send reports through various communication channels (here as an [email](doc:email)): [block:code] { "codes": [ { "code": "host \"my-app-host\", hostname: \"host.example.com\"\n\njob \"my-db-backup\" do\n resource \"database\", name: \"my-database\", host: \"my-app-host\"\n remotely as: \"user\" do\n # Create a MySQL dump\n mysql_dump user: \"mysql-user\", password: \"secret\"\n # Put the dump in an archive and delete the dump\n tar_gz delete_source: true\n end\n # Move the archive on the localhost\n move to: \"localhost\", using: \"scp\", as: \"user\"\n # Place an additional copy into the Cloud using Amazon S3\n copy to: \"my-bucket/my-project\", using: \"s3\", access_key_id: \"XXX\", secret_key: \"secret\"\n notify via: \"email\", using: \"sendmail\", from: \"robots@example.com\", to: \"me@example.com\"\nend", "language": "ruby" } ] } [/block] ## Configuring remote access ES.IO relies on the SSH protocol to execute commands remotely (and to copy resources when using the SCP transport). By default, it uses the SSH key located at `~/.ssh/id_rsa`. A good practice it to create and use a specific keypair so that you can easily add/remove it to/from remote hosts. To do so, create an SSH keypair and deploy the public key to the remote host(s): [block:code] { "codes": [ { "code": "ssh-keygen -t rsa -N '' -f ~/.ssh/electric_sheep_rsa\nssh-copy-id -i ~/.ssh/electric_sheep_rsa.pub user@host.example.com", "language": "shell", "name": "SSH Setup" } ] } [/block] You can specify the private key to use for authentication for each host: [block:code] { "codes": [ { "code": "host \"my-app-host\", hostname: \"host.example.com\", private_key: \"~/.ssh/electric_sheep_rsa\"", "language": "ruby", "name": "Sheepfile" } ] } [/block] You can specify the specific ssh port to use for the connection for each host: [block:code] { "codes": [ { "code": "host \"my-app-host\", hostname: \"host.example.com\", ssh_port: 22222", "language": "ruby", "name": "Sheepfile" } ] } [/block] [block:callout] { "type": "warning", "title": "Empty passphrase", "body": "So far, ElectricSheep.IO does not support agent-based authentication or passphrases." } [/block] ## Running ElectricSheep.IO ### Standalone Simply run ES.IO from the directory containing your _Sheepfile_: [block:code] { "codes": [ { "code": "electric_sheep", "language": "shell", "name": "Run" } ] } [/block] ES.IO will process backup jobs sequentially and exit. ### As a daemon Start a master process from the directory containing your _Sheepfile_: [block:code] { "codes": [ { "code": "electric_sheep start", "language": "shell" } ] } [/block] The master process will look at jobs scheduling and fork to process the backup jobs when needed. By default, the master will read and write the _Sheepfile_, PID file and log file in the current directory. See the [Command Line Reference](doc:command-line) for startup options. If you want to place the master process in the background and give control back to the terminal: [block:code] { "codes": [ { "code": "electric_sheep start -d", "language": "shell" } ] } [/block]