Capistrano recipe: update your remote shared assets and database to your local application
Update: rubyphunk created a more robust solution for this, check out his reflection gem for keeping your development machine in sync.
When working on a production site there is often a need to get the latest data from the database or the latest shared assets onto your local development machine.
Shared Assets are typically items that users uploads through some interface that you don’t want to store in your git repository but don’t want to overwrite on deploy. For example you have a user model and people can upload their avatar. We don’t want that image file in the repository but we do want to keep it on the live site.
When I talk about shared assets I mean specifically items in the public directory. These items should be in your .gitignore file as well as a symlink that is setup in your deploy recipe. A directory like public/uploads or public/images/photos. You shouldn’t be using this for javascript files, css, regular images, etc.
The live database often contains data that modifies either the content or display of your site. Various items such as blog posts, recent messages, and many other items.
Wouldn’t it be nice if you could have an easy way to make your local development site look just like the production site?
Assuming you already have a functioning deploy recipe the following is an additional namespace you can add to have this functionality. Also, make sure that you have your keys on your remote server so it doesn’t prompt you to login.
First lets setup the shared host that we want to pull this information from. We could probably pull from one of the roles but for now we are just going to use a single shared host. Modify this recipe as you see fit, this is just more of a base for people to work off of.
Specify the shared host to pull your remote assets and database from
set :shared_host, "nickhammond.com"
Now lets setup the namespace and recipe for the two tasks that we want to accomplish.
namespace :update do desc "Mirrors the remote shared public directory with your local copy, doesn't download symlinks" task :shared_assets do if shared_host # Used friendly options so it's easier to read # I'm using rsync so that it only copies what it needs # Windows users you can use the download method within capistrano and pass recursive => true run_locally("rsync --recursive --times --rsh=ssh --compress --human-readable --progress #{user}@#{shared_host}:#{shared_path}/public/ public/") else puts "Ummmm. You should define a shared_host variable so I know where to get the files from." end end desc "Dump remote production database into tmp/, rsync file to local machine, import into local development database" task :database do # First lets get the remote database config file so that we can read in the database settings get("#{shared_path}/config/database.yml", "tmp/database.yml") # load the production settings within the database file remote_settings = YAML::load_file("tmp/database.yml")["production"] # we also need the local settings so that we can import the fresh database properly local_settings = YAML::load_file("config/database.yml")["development"] # dump the production database and store it in the current path's tmp directory. I chose to use the same filename everytime so that it would just overwrite the same file rather than creating a timestamped file. If you want to use this to create backups then I would recommend putting something like Time.now in the filename and not storing it in the tmp directory run "mysqldump -u'#{remote_settings["username"]}' -p'#{remote_settings["password"]}' -h'#{remote_settings["host"]}' '#{remote_settings["database"]}' > #{current_path}/tmp/production-#{remote_settings["database"]}-dump.sql" # If your database is large you might want to bzip it up and then extract it later # run_locally is a method provided by capistrano to run commands on your local machine. Here we are just rsyncing the remote database dump with the local copy of the dump run_locally("rsync --times --rsh=ssh --compress --human-readable --progress #{user}@#{shared_host}:#{current_path}/tmp/production-#{remote_settings["database"]}-dump.sql tmp/production-#{remote_settings["database"]}-dump.sql") # now that we have the upated production dump file we should use the local settings to import this db. run_locally("mysql -u#{local_settings["username"]} #{"-p#{local_settings["password"]}" if local_settings["password"]} #{local_settings["database"]} < tmp/production-#{remote_settings["database"]}-dump.sql") end end
Looks pretty good, will give it a try later. Thank you!
I’m so glad to have stumbled upon this. I was looking for a nice solution to this for a while; very useful for many of my projects. Thanks!
Thanks for the slick script, I wrote something similar to this, but had some issues with getting the remote DB vars. This is a nice and clean implementation! I think I am going to include it in a Capistrano recipe gem I am writing if you don’t mind.
Thanks for sharing!
It is nice however coming from a network/system admin there is one major flaw and that is security. Running mysqldump with -ppassword is a very discouraged method. At this point in time I do not have a better suggestion other than using configuration files (one for each database/site) however I can't say that it is a good idea to do that either especially in my case where 30-50+ database passwords are in plain text files.
I also prefer to set variables for example remote_db_name once then re-use that variable, it just helps with maintenance down the road. Instead of re-typing remote_settings["database"] over and over especially using interpolation it just gets messy. Between using short options for rsync command and combining some of the pathing into variables I was able to get all the commands onto one line each.
Hopefully some day we will figure out how to securely access local and remote databases using mysql via Capistrano, but again as far as I know this or config files (.my.cnf) is as good as it gets.
Nice article
thanks a lot!
I just replace a “& gt;” and “& lt;” for “>” and “<" after copying and also removed "-h'#{remote_settings["host"]}'" because we using a socket connection on production. And it start working!
ooh, ad also I had add a new task for convinient to do both actions:
task :all do
update.shared_assets
update.database
end