Backups and Backup Restoration

As your Forgejo installation becomes a critical part of your development lifecycle, the data within it needs to be saved in a backup in order to ensure you don’t lose anything.

Forgejo doesn’t provide or recommend any specific backup tool, but does provide information on what needs to be backed up, how to manage a backup cycle, and how to restore Forgejo to working order from a backup.

Backing Up Forgejo

Warning

It is strongly recommended to shut down Forgejo during a backup.

Forgejo saves data in multiple different systems, even in its simplest configuration: Git repositories and bulk files are saved on disk as regular files, and a SQL database is used to record information about user interactions. It is not trivial to perform a consistent backup of all that data while Forgejo is running. An inconsistent backup may result in data loss when attempting to restore it.

Database

For SQLite, the database file in the config option [database].PATH needs to be backed up with any file backup tool.

For other databases, use database backup tools that are appropriate to your selected database engine to create a backup of the Forgejo SQL database. Any tool designed to work with that database engine should work with Forgejo.

Note

With a SQLite database and a default configuration, backing up the data folder covers the requirements described in the Database, Repositories, Storage, Queues, and Indexer sections.

Repositories

Forgejo’s repositories are stored on disk. By default, they are in the directory data in Forgejo’s WORK_PATH, in a subdirectory forgejo-repositories. This configuration may be changed with either [server].APP_DATA_PATH or [repository].ROOT.

Once you’ve identified the correct folder, you may use any file backup tool to backup this folder when Forgejo is offline.

Storage

Forgejo’s storage subsystem is used for the storage of larger files. Each supported subsystem can be configured to store data in local files (called local), or in an S3-compatible object storage system (called minio).

If using local storage, then backing up Forgejo must include all the configured subsystem directories. By default, all of these paths are under the directory data in Forgejo’s WORK_PATH, but this can be overridden with [server].APP_DATA_PATH, or individually on each storage subsystem. Check that every expected path in the storage subsystem list is in your backup, if they’re applicable to your installation. You may use any file backup tool to backup the locations when Forgejo is offline.

If using minio storage, then you should make sure that the remote storage system has appropriate backups, or use a backup tool to keep a copy of the stored artifacts.

Queues

Background work that is queued but has not yet been executed is stored in a configurable queuing system, defined by the config settings under the [queue] section. If this data is lost, then queued work will never be completed, which will cause system inconsistencies. These inconsistencies would generally recover over time as new background work replaces the lost work.

If the [queue].TYPE setting is level, which is the default value, then by default the queues are in the directory data in Forgejo’s WORK_PATH, in a subdirectory queues. This configuration may be changed with either [server].APP_DATA_PATH or [queue].DATADIR. You may use any file backup tool to backup the locations when Forgejo is offline.

If the [queue].TYPE setting is redis, then ensure the connected redis server implementation is persistent (not in-memory only), and is backed up.

For the [queue].TYPE of channel or dummy, no backups are possible. Queued background work will be lost when Forgejo is shutdown.

Configuration Settings

The Forgejo configuration file, app.ini, contains a few critical values that must be backed up in order to restore an instance later. It is recommended that the entire file is backed up, but pay specific notice to these configuration settings which contain unique, non-reproducible information:

  • [security].SECRET_KEY, or the file located at [security].SECRET_KEY_URI: some values are encrypted in Forgejo’s database using this key. If it is not preserved between a backup and a restore, then Forgejo will not be able to access the encrypted values. This will break credentials stored in Forgejo, such as webhook credentials, Forgejo Actions secrets, repository mirroring credentials, two-factor authentication configurations, and possibly more in the future.
  • [oauth2].JWT_SECRET, or the file located at [oauth2].JWT_SECRET_URI: OAuth credentials generated by Forgejo are encrypted or signed with this key. If it is not preserved between a backup and a restore, then systems integrating with Forgejo will need to reauthenticate.

Indexers

Indexers provide the ability to search issues, or search repository code. If the data related to them is lost, then it can be rebuilt, but it may be time-consuming or I/O intensive. Backing it up is recommended for large installations.

Issue Indexer

If the [indexer].ISSUE_INDEXER_TYPE setting is bleve, which is the default value, then bleve data by default is in the directory data in Forgejo’s WORK_PATH, in a subdirectory indexers. This configuration may be changed with either [server].APP_DATA_PATH or [indexer].ISSUE_INDEXER_PATH. You may use any file backup tool to backup the locations when Forgejo is offline.

If [indexer].ISSUE_INDEXER_TYPE is db, then the database is used for issue searches. No additional data must be backed up.

If [indexer].ISSUE_INDEXER_TYPE is elasticsearch or meilisearch, then the connected search engine should be backed up independently of Forgejo.

Repository Indexer

If [indexer].REPO_INDEXER_ENABLED is set to true, then repository code indexing will occur and that data may be backed up. If not, this section can be skipped.

If the [indexer].REPO_INDEXER_TYPE setting is bleve, which is the default value, or zoekt, then index data by default is in the directory data in Forgejo’s WORK_PATH, in a subdirectory indexers. This configuration may be changed with either [server].APP_DATA_PATH or [indexer].REPO_INDEXER_PATH. You may use any file backup tool to backup the locations when Forgejo is offline.

If [indexer].REPO_INDEXER_TYPE is elasticsearch, then the connected search engine should be backed up independently of Forgejo.

Logs

It is not necessary to backup Forgejo’s logs. However, you may choose to do so if they are used routinely for log analysis, or may form an important chain analyzing a security incident in the future.

If [log].MODE is set to file, then [log].ROOT_PATH will contain Forgejo’s server logs. You may use any file backup tool to backup this location when Forgejo is offline.

If [log].MODE is set to console, then Forgejo’s logs are provided to another system (for example, systemd, or docker). Backing up these logs would require specific knowledge of the external system.

Restoring Forgejo

Restoring Forgejo from a backup requires coordinating the knowledge of how it was backed up, what tools were used to back it up, and what configuration options were in-use in Forgejo.

In general:

  • Restore Forgejo’s database, using the appropriate database-specific tool, or file-based tool if using SQLite.
  • Restore files, as they were backed up. Typically this is as simple as restoring a data directory, which is the parent of all the default file locations. The file locations are described above: Repositories, Storage, Queues, and Indexers.
  • Restore the app.ini config file, or rebuild it if necessary pointing to new restored file paths.
  • Start-up Forgejo.

After successfully starting Forgejo and logging in as an administrator, these maintenance tasks can be useful to ensure the Forgejo instance is fully operational after a restore. From the /admin/monitor/cron admin management page:

  • If using SSH: Regenerate Forgejo SSH keys and SSH principals by running the “Update the “.ssh/authorized_keys” file with Forgejo SSH keys” and “Update the “.ssh/authorized_principals” file with Forgejo SSH principals” cron jobs. These files were not included in the backup procedures described as all the data in the database is authoritative and these files can be recreated easily.
  • If you did not backup, or restore, the issue indexer: Rebuild the issue index by running the “Rebuild issue indexer” cron job.
  • If you did not backup, or restore, the repository indexer, it will rebuild automatically on Forgejo startup.