Losing any amount of data can disrupt business in significant ways: it can erase history, lead to regulation violations, and even bankrupt a business. Wholly apart from server outages and natural disasters, it’s not uncommon for users to make mistakes while handling data. They could accidentally delete information from a repository, rogue employees could destroy data, accounts might get deleted, not to mention malicious hijackers could compromise your data.
With so many unforeseeable scenarios that could lead to data loss, this is why backups are a necessity. Repositories store information crucial to modern applications, such as code, documentation, and work history. Whichever repository platform you use, it’s vital to maintain regular backups.
Unfortunately, in the case of Bitbucket, your data is not easily accessible for restore. Although Bitbucket does an automatic backup of all repositories hosted, it does not provide restore access to users. Hence the need to back up repositories on your own without having to depend on Bitbucket support. Backing up your Bitbucket repository on your own gives you the flexibility to back up and restore your data at any time and in any way you want.
This guide will introduce you to three different tools useful for the backup and restore of your repositories in Bitbucket Cloud. You’ll learn how to use them, as well as the pros and cons of each method.
Method 1: Using git clone Command-line Script
Bitbucket does not provide a built-in backup feature. Currently, the simplest way to back up a repository in Bitbucket is to download it manually using the git clone command. Every clone of a repository is a backup.
The git clone command is a Git command primarily used to point to an existing repository, in this case your Bitbucket repository. It clones or makes a copy of your repository and downloads it to your computer. You will then have an exact local copy of your repository just like it is on the Bitbucket site.
To use this command, you must have Git installed on your terminal:
bash
git clone <repository_url> <directory>
In the sample above, <repository_url> represents the link to your Bitbucket repository. The <directory> command is an optional argument that points to the path of the folder you want the repository downloaded to. <directory> defaults to your current folder.
The advantage of using this backup method is that it’s simple to use and can be run from any computer with a terminal. There’s also the flexibility of backing up on several computers and other cloud options.
Its major drawback is that it’s a very manual process. The repo owner is responsible for making sure that clones are made periodically so as to have up-to-date backups. Having to back up manually undermines best practices of backing up data, sometimes to the point of defeating the purpose. Also, the backups do not include the bells and whistles of a Bitbucket repository, such as pull requests, comments, and issues.
To learn more about the git clone command, check out the git clone documentation.
Method 2: bbbackup
bbbackup is a free open-source library used as a backup solution for cloud-to-local repository cloning. It’s written in Python, which means to use it you need to have Python installed on your machine.
To start using bbbackup, you need to first clone the tool’s repository to your local computer by running:
bash
git clone git@github.com:OpenReplyDE/bbbackup.git
cd bbbackup
bbbackup requires the use of a config file called bbbackup.cfg. This file will contain your Bitbucket credentials and other configurations.
The repository comes with a sample config file called bbbackup_sample.cfg. Rename it to bbbackup.cfg. Open the file and edit the authorization config as auth_method = uidpwd, since you are going to be using the password method to authenticate your Bitbucket account.
In the [bitbucket_account] section, fill the userid. This should be the normal Bitbucket username of the account containing your repositories. For the password variable, you might need to create an app password in Bitbucket, if you don’t already have one. To create an app password, check the Bitbucket documentation on creating app passwords. The teamname variable refers to the user/organization name being backed up.
In the [backup] section, the filepath variable represents the name of the folder you want your Bitbucket backups to be stored in. Make sure you include a full path to the folder.
To start using bbbackup, create a virtual environment with Python 3:
bash
python3 -m venv BitBucketBackup
Next, activate the virtual environment, upgrade pip to its latest version, and install the Python libraries required:
bash
source bin/activate
pip install --upgrade pip
pip install -r requirements.txt
Now, test that the backup script is working by running:
bash
python bbbackup.py --help
Because the bbbackup script is going to back up the repositories by using SSH, make sure that you have set up an SSH key for your Bitbucket account before moving on.
To start the backup process, run:
bash
python3 bbbackup.py --configuration bbbackup.cfg --notify --backup
The above command will back up all of your Bitbucket repositories to the folder you specified in the filepath section of the config file. The backed-up repositories are stored in the backup folders in a plain state, so in this case restoring the repository to Bitbucket is as simple as running a git push command.
bbbackup is a great tool for backing up your Bitbucket repository, a step up from simply using git clone manually for every single repository in your Bitbucket account. With bbbackup, you can back up all your repositories in one fell swoop.
Note that it supports automation only on Unix machines. This is because it includes a bbbackup.sh script, which is prepared to be called by a cron job or macOS daemon. It also includes support for reporting backup progress via Slack.
A major con of using bbbackup is that it does not provide an option to exclude certain repositories from the backups. Another disadvantage is that the backups do not include pull requests, comments, and issues.
Method 3: SCM Backup
SCM Backup is a free open-source backup tool used offline for backing up repositories. It supports backing up from several online hosting platforms, like Bitbucket and GitHub. It uses the hosting platform’s API to list and fetch your repositories, then uses the version control system installed on your machine to clone the latest data of every repository into a local backup folder. It also supports hosting and backing up multiple users/teams per hosting platform.
SCM Backup is a framework dependent tool, requiring the use of .NET Core. It’s been written and tested only on the Windows platform for that reason. To demonstrate the use of SCM Backup, this guide will use Docker to avoid cross-platform errors.
SCM Backup requires a settings.yml file that contains relevant information like your hosting platform, repository name, workspace ID, and authentication details. Create a folder called scm_backup and in it create a settings.yml file:
bash
mkdir scm_backup && cd scm_backup
touch settings.yml
Add the following code to the settings.yml file:
yml
localFolder: '/scm-backup-out'
sources:
- title: backup_from_bitbucket
hoster: bitbucket
type: org
name: <workspace_id>
authName: <username>
password: <app_password>
The localFolder value should represent the name of a directory where you want all the backed-up repositories to be stored. But since we are using Docker, the backups will be stored inside a folder, /scm-backup-out, in the Docker container.
In sources, title represents the title of your backup. It must be a unique name in the file because you can have several sources with their own titles. hoster represents the hosting platform name, in this case Bitbucket.
type refers to whether you are backing up on the account of a single user or an org. The values could be user or org. name refers to the Bitbucket user or organization name being backed up.
Your authname refers to the normal Bitbucket username of the Bitbucket account. For the password, you might need to create an app password in Bitbucket. To create an app password, check the Bitbucket documentation on creating an app password.
SCM Backup supports excluding repositories from the backup. To use that feature, add the following to your settings.yml file:
ignoreRepos:
- repo1
- repo2
The values repo1 and repo2 represent your repository name. You can list as many repositories as you want for exclusion from backup.
Create a file named Dockerfile and add the following code:
docker
FROM ubuntu:18.04
ENV DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true
WORKDIR /scm-backup
# install .NET core and SCM Backup
RUN apt-get update && apt-get -y install software-properties-common &&
add-apt-repository universe && apt-get update &&
apt-get -y install sudo wget unzip apt-transport-https git &&
wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb &&
dpkg -i packages-microsoft-prod.deb && rm packages-microsoft-prod.deb &&
apt-get update &&
apt-get -y install dotnet-sdk-3.1 &&
wget https://github.com/christianspecht/scm-backup/releases/download/1.4.0/scm-backup-1.4.0.ae7edc3.zip &&
unzip scm-backup-1.4.0.ae7edc3.zip
# copy settings.yml from local filesystem to the container
COPY settings.yml .
# run the backup
ENTRYPOINT dotnet ScmBackup.dll
In the Dockerfile, an Ubuntu environment is created, the working directory is changed to scm_backup, and .NET Core and SCM Backup tools are downloaded. Next, the settings.yml config file is copied into the container directory, and the SCM Backup tool is run.
Make sure Docker is running, then build the Docker image using the docker build command:
bash
docker build -t scm-backup .
Before running the container, we need to create a folder outside the container where Docker will transfer the Bitbucket backups. You can name it bitbucket-backups.
bash
mkdir bitbucket-backups
Now run the container:
bash
docker run -t --mount src=`pwd`/bitbucket-backups,target=/scm-backup-out,type=bind scm-backup
Check the bitbucket-backups folder created above, and you should see the _config folder. It contains a backup of your current settings and the backup_from_bitbucket folder, which contains all the backed-up repositories.
Restoring a Repository with SCM Backup
To restore a repository backup into a working directory, navigate into bitbucket-backups and from there navigate into the directory of one of your backed-up repositories. In it you should see folder repo and/or a folder wiki, if the repository has a wiki.
Restore by running the code below, where <repo_name> refers to the local path and name you want the repository restored to on your machine:
bash
git clone repo </path/repo_name>
Now you should see your restored repository as a working directory in /path/repo_name.
SCM Backup is a great option for backing up your Bitbucket repositories because it supports other hosting platforms and version control systems, not just Bitbucket and GitHub. It also supports excluding repositories you don’t want to back up. It is simple to use and easy to restore backed-up repositories.
A major con of using it, however, is that it’s another manual process and doesn’t support automatic backup. It also does not back up parts of a repository like pull requests, comments, and issues.
Conclusion
Since Bitbucket itself has not introduced a way to back up repositories, it’s important to be able to back up and restore your repository data on your own. In this guide, you learned three different methods for backing up a Bitbucket repository that don’t depend on the hosting platform.
All of these methods require that you write your own command-line script or manually run them. They’re not fully automated or monitored for daily backups. None of them back up to third-party locations by default (e.g. Amazon S3), so you’ll have to write a script to move files. They also don’t pull added data that accompanies repositories, such as pull requests, issues, and comments.
Sign up today for Rewind Backups for Bitbucket in the Atlassian Marketplace for automatic daily backups with Rewind, the leading repository backup solution.
Adeyinka Adegbenro">
