Differences

This shows you the differences between two versions of the page.

--- general:disk-space-management [2017/02/26 23:28]
mscherf [Backups] added note about site plugins for backups
+++ general:disk-space-management [2017/07/07 15:55] (current)
mscherf revisions for #155
@@ Line 7: / Line 7: @@
 Although backups do themselves take disk space, it is still very important to have them. To protect against data loss, it is always strongly recommended not only to keep multiple sets of backups, but also to keep copies of at least some of them in an offsite location. Although we do create our own [[vps-hosting:information-about-vps-backups-and-restorations|backups of the VPSs]], there are multiple reasons it is important not only for you to create your own backups as well, but also to keep at least one of the copies offsite.
-For cPanel/WHM servers, you can read more details about [[https://documentation.cpanel.net/display/ALD/Backup+Configuration|configuring automatic backups]] or [[https://documentation.cpanel.net/display/ALD/Backup+Wizard|manually triggering backups at the time you want them]]. For DirectAdmin servers, you can configure backups from [[https://help.directadmin.com/item.php?id=269|any of the three access levels]] and it is possible to [[https://help.directadmin.com/item.php?id=511|store the generated backup both locally and remotely]]. For servers still using Plesk,((Plesk is no longer available for new servers)) you can learn more about the backup system [[http://docs.plesk.com/en-US/12.5/advanced-administration-guide-linux/backing-up-restoring-and-migrating-data/backing-up-data.68837/|here]].
+For cPanel/WHM servers, you can read more about how to configure automatic backups and some of our recommendations for them [[control-panels:cpanel-whm:backup-configuration|here]]. For manually triggering backups at the time you want them via cPanel, you can read more [[https://documentation.cpanel.net/display/ALD/Backup+Wizard|here]]. For DirectAdmin servers, you can configure backups from [[https://help.directadmin.com/item.php?id=269|any of the three access levels]] and it is possible to [[https://help.directadmin.com/item.php?id=511|store the generated backup both locally and remotely]]. For servers still using Plesk,((Plesk is no longer available for new servers)) you can learn more about the backup system [[http://docs.plesk.com/en-US/12.5/advanced-administration-guide-linux/backing-up-restoring-and-migrating-data/backing-up-data.68837/|here]].
-If you have a Dedicated Server, it is likely that automatic backups are already enabled in WHM, since for Dedicated Servers we enable that by default. But even on the Dedicated Servers, we cannot automatically configure remote Additional Destinations, since you would need to provide some of the [[https://documentation.cpanel.net/display/ALD/Backup+Configuration#BackupConfiguration-AdditionalDestinations|relevant information]]. Additionally, when accounts are migrated into a server, they keep their backup settings (that is, whether backups are enabled/disabled) from the old server.
+<WRAP important>
+It is particularly strongly recommended **not** to use site plugins or similar to generate backups. Very frequently this type of site plugin runs at an unnecessarily high i/o priority, which in many cases can use up all the disk time and prevent other processes (such as mysql or apache) from having quick access to the disk. For this reason, this type of plugin is extremely likely to slow down all of the sites on your server, and they are not recommended.
+</WRAP>
 On the Dedicated Servers, the backups are by default configured to be stored on the secondary disk, so even in the event that the partition for the backups becomes completely full, it will not interfere with the functionality of the daemons (though it would cause those backup runs to fail, and those backup files to become corrupted). But on a VPS, there is only a single partition, so it is very important to make sure the backups being generated and stored there will not cause the disk to become completely full.
-It is not recommended to include backups inside other backups. For example, if there is a site plugin making backups, and then Softaculous makes backups of the site which contain the backups made by the plugins, then on top of that WHM is making backups of the cPanel account which contain the Softaculous backups (which, as mentioned, also in turn contain the backups made by the plugin), then each backup set will cause multiple times the amount of diskspace to be taken as otherwise. If redundancy is desired (which is recommended), then rather than having nested backups, choose one backup type, and then run that type more often, retain more sets, and/or copy the sets to more locations. If you are sure you would like to use multiple backup systems simultaneously, it is strongly recommended to exclude each type of backup from the backups generated by the other systems. You can learn more about excluding files from cPanel backups [[https://documentation.cpanel.net/display/CKB/How+to+Exclude+Files+From+Backups|here]].
+Overall, when choosing a backup configuration, it is important to consider all of the following factors:
-<WRAP important>It is particularly strongly recommended **not** to use site plugins or similar to generate backups. Very frequently this type of site plugin runs at an unnecessarily high i/o priority, which in many cases can use up all the disk time and prevent other processes (such as mysql or apache) from having quick access to the disk. For this reason, this type of plugin is extremely likely to slow down all of the sites on your server, and they are not recommended.</WRAP>
-Because sometimes the days chosen for weekly and monthly backups coincide, it is recommended to retain at least two of at least one of the three frequencies.((ie, retain at least two daily backups, or at least two weekly backups, or at least two monthly backups)) For example, if you only retain one each of daily, weekly, and monthly backups, then whenever the chosen day of the week for the weekly backups happens on the first or fifteenth of the month, you will only be retaining one backup that time, resulting in a single point of failure. If for some reason on that day backups fail to complete, you will not have any backups at all until they run again.
-So overall, when choosing a backup configuration, it is important to consider all of the following factors:
   * How much disk space you have available in the server
@@ Line 26: / Line 22: @@
   * How long ago you want to be able to revert accounts to
-If you're not sure, you may want to start with a basic configuration as follows:
+For more specific recommendations,((for cPanel/WHM servers)) please read more [[control-panels:cpanel-whm:backup-configuration|here]].
-  * Global Settings:
-    * Backup Status: Enable
-    * Backup Type: Compressed
-  * Scheduling and Retention:
-    * Backup Daily: Enable
-      * Select days
-      * Retain (1 or 2) Daily Backups
-    * Backup Weekly: Enable
-      * Select day
-      * Retain (1 or 2) Weekly Backups
-    * Backup Monthly: Enable
-      * Select day(s)
-      * Retain (1 or 2) Monthly Backups
-  * Files:
-    * Backup Accounts: Enable
-      * Select Users: make sure all users on this page are set to "On" in the "Backups" column
-    * Backup System Files: Enable
-  * Databases:
-    * Per Account and Entire MySQL Directory
-  * Configure Backup Directory:
-    * Retain backups in the default backup directory: Enable
-    * Mount Backup Drive as Needed: Disable (if VPS)
-  * Additional Destination: Add information as appropriate for each additional destination desired
-Notes:
-  * These settings are written for a cPanel server. For Plesk or DirectAdmin, similar equivalent settings are recommended where possible.
-  * Retain at least two backups of at least one of the three types (daily/weekly/monthly)
-  * It is **strongly recommended** to have //at least// one "Additional Destination" configured pointing to a remote destination.
-  * It is recommended to only use Additional Destinations that can use secure, encrpyted connections.((eg, Amazon S3, SFTP, or WebDAV with SSL Enabled))
-  * If there is not enough disk space within the server to store the backups, then once you have confirmed that the remote Additional Destination is working, you can deselect (disable) the option "Retain backups in the default backup directory".((in section "Configure Backup Directory"))
-  * There needs to be enough disk space in the server to //generate// the backups, even if they are not being stored in the server. For this purpose, you should ensure you have at least twice the size of the largest account free, plus an extra gigabyte. If you do not have this much disk space in your server, it is strongly recommended to add more disk space to your server, so that you have adequate room to generate backups.
-  * For convenience, it is recommended if possible to store the backups in the server as well as at the remote destination. For this, it is necessary to have at least twice the amount of free space as is used by one set of backups, since old backups are not deleted until the new backups have finished being generated.
-  * This example would require approximately((the exact amount of space needed can vary, depending on several factors, but this is an approximation)) space to store four sets of backups, plus space for two sets of backups during backup generation (the one being generated, plus the one that is soon going to be deleted), plus 2 * (largest account size) + 1GB for the account backup currently in progress. (This is not including the disk space required for the accounts themselves or the system files for the sites and services to run.)
-The backup settings can be changed later if needed.
 ===== Account Quotas =====
@@ Line 125: / Line 84: @@
 The output will appear similar as in the above example, but with data for ''%%/backup%%'' instead of ''%%/%%''.
+If you want to check how many backups are present and how big they are, you can check with this command:
+<code>
+root@host [~/support]# find /backup -type d -name accounts -print0 | xargs -0 du -sch | sort -h
+.0K	/backup/2017-07-02/accounts
+.0K	/backup/2017-07-03/accounts
+.0K	/backup/2017-07-04/accounts
+.0K	/backup/2017-07-05/accounts
+.0K	/backup/2017-07-06/accounts
+.0K	/backup/2017-07-07/accounts
+.0K	/backup/monthly/2017-07-01/accounts
+.0K	/backup/weekly/2017-06-25/accounts
+.0K	/backup/weekly/2017-07-02/accounts
+.0G	/backup/monthly/2017-06-01/accounts
+.0G	/backup/monthly/2017-06-15/accounts
+G	total
+</code>
+In this case, a Remote Destination was configured, and later the server was configured not to retain the backups locally. The backups that were already present before that were not removed, but all subsequent ones were deleted from the server after successful transfer to the remote destination.
+Although this only looks for the account backups, these are usually larger than the system backups.
+If you decide you want to remove one or more of the backups to save space, you can use the `cd` command to go to the desired folder, and then use the `rm -v` command to remove the specific files you would like to remove.
 After checking the total disk space usage of the server, it is useful to find how much is being used by each account. You can get a general overview using ''%%repquota%%'':
@@ Line 156: / Line 139: @@
 <code>
 root@host [~]# cd /path/to/folder/
-root@host [/path/to/folder]# du -h --max-depth=1 .
+root@host [/path/to/folder]# du -hd 1 .
 K	./.cphorde
 K	./etc
@@ Line 184: / Line 167: @@
 <code>
-root@host [/path/to/folder]# du -h --max-depth=1 . | sort -h | tail
+root@host [/path/to/folder]# du -hd 1 . | sort -h | tail
 K	./.razor
 K	./.cpanel
@@ Line 200: / Line 183: @@
 <code>
-root@host [/path/to/folder]# du -ha --max-depth=1 .
+root@host [/path/to/folder]# du -had 1 .
 K	./.cphorde
 .0K	./.contactemail
@@ Line 243: / Line 226: @@
 <code>
-root@host [/path/to/folder]# du -ha --max-depth=1 . | sort -h | tail
+root@host [/path/to/folder]# du -had 1 . | sort -h | tail
 K	./.razor
 K	./.cpanel
@@ Line 268: / Line 251: @@
 <code>
-root@host [/path/to/folder]# du -h --max-depth=1 . | sort -h | tail
+root@host [/path/to/folder]# du -hd 1 . | sort -h | tail
 K	./.razor
 K	./.cpanel
@@ Line 284: / Line 267: @@
 <code>
-root@host [/path/to/folder]# du -ha --max-depth=1 . | sort -h | tail
+root@host [/path/to/folder]# du -had 1 . | sort -h | tail
 K	./.cpanel
 K	./.cphorde
@@ Line 309: / Line 292: @@
 ==== Temporary Files ====
-Usually temporary files will not be the main cause of the server running out of disk space, but if it is so full you need to use the console instead of regular ssh, it is possible that removing files from ''%%/tmp%%'' might, depending on the severity of the issue, clear enough space for ssh to start working. It is very important //**not**// to remove the folder itself, or any socket files or subdirectories that might be present. In a cPanel server, this command is likely to be safe:
+Usually temporary files will not be the main cause of the server running out of disk space, but if it is so full you need to use the console instead of regular ssh, it is possible that removing files from ''%%/tmp%%'' might, depending on the severity of the issue, clear enough space for ssh to start working. It is very important //**not**// to remove the folder itself, or any socket files or subdirectories that might be present. Before removing the files, check how much space they are taking up:
 {{howhard>5}}
+<code>
+# find /tmp -type f -print0 | xargs -0 du -sch | sort -h | tail
+</code>
+If the files in ''%%/tmp%%'' are large enough to make a difference, you may want to consider removing them. To do so, in a cPanel server, this command is likely to be safe:
 <code>
@@ Line 344: / Line 333: @@
 <code>
-# find /home*/*/.trash -depth -mindepth 1 -exec rm -rv +
+# find /home*/*/.trash -depth -mindepth 1 \( \( ! -type d -exec rm -v {} + \) -o \( -exec rmdir -v {} + \) \)
 </code>
@@ Line 413: / Line 402: @@
 In this case, the largest file manager trash folder was smaller than the smallest imap trash folder.
+Similarly, if you want to empty all of the trash folders at once, you can do so as follows:
+<code>
+# find /home*/*/{.trash,mail/{*/*/,}.Trash} -depth -mindepth 1 \( \( ! -type d -exec rm -v {} + \) -o \( -exec rmdir -v {} + \) \)
+</code>
 And remember, if you need any help, you can always [[my-knownhost:howto-login|open a Support Ticket]] for assistance.

KNOWNHOST WIKI

User Tools

Site Tools

Differences

Page Tools