Planning a Migration

Introduction > Edge Filer Planning > Migrating a File Server to a CTERA Edge Filer > Planning a Migration

When Planning the migration, CTERA recommends creating a spreadsheet with the following data:

Shares Names – The names of each share to migrate along with the following information:

Folders/Paths

Size

Number of Files

Average file size

Shares security ACL – Enabled in the CTERA Portal when defining each folder.

This information is used to select the optimal deduplication ratio and block size for the setup.

In addition, configure the file server as follows:

Limit each folder/share based on the block size.

	Maximum
Block Size	Cloud Folder Size	Folder Group Size
512KB	16TB	64TB
1MB	32TB	128TB
2MB	64TB	128TB
4MB	128TB	128TB

Note: Guidelines to calculate optimal block sizes are available from CTERA.

Limit deduplication levels to 128TB or 128,000,000 files, regardless of block sizes. CTERA recommends setting the deduplication levels for the cloud drive in the virtual portal settings, under Default Settings for New User to Folder and a folder group for each of the user account's devices, containing all of the device's cloud drive folders. Deduplication is performed separately for each of the user account's folder groups.

Note: Some operations can only be performed in the CTERA Gateway user interface:

Share creation and management.

Share permissions.

Share Architecture

The proposed architecture assumes you wants to keep the same shares configuration as in the original file server. For this share configuration, you map all the users to the top root of the Windows servers and all the shares are configured with team-based permissions.

For example, the following team folders exist on a Windows server named SRV2:

You create the corresponding folders in the CTERA Portal and enable Windows ACLs and set the collaboration required for each folder and then sync them down to the CTERA gateway. Once the folders are synced to the CTERA gateway you need to create the shares pointing to these folders via the CTERA Gateway user interface. The folder structure is critical to a smooth migration.

For example, where appropriate, CTERA recommends creating site or company cloud folders. For example, if the migrated data will be used at multiple company sites, with similar share structures at each site, you should create a corporate cloud folder with all the folders that will be shared by all the sites underneath it and in addition have a site folder per site containing all the share folders for the site.

With this structure, you can edit the ACLs for each folder at the top level of the share and if you use a central gateway for failover or performing restores, it is clear which dataset you are working with. In the gateway the shares are defined with Windows ACL Emulation Mode, as in the following example.

For details about setting up the shares, refer to the Gateway Administration.

Note: The backups and public shares are default shares created by CTERA. For details about how to hide the default shares, refer to the Gateway Administration.

The view from Windows Explorer on the CTERA Gateway now matches the view from the original server.

Migrating Using One or More Ingestion Gateway

As with any migration, the aim is to successfully complete the migration with minimum downtime for end users. CTERA recommends using up to four CTERA gateways in Caching mode as dedicated Ingestion devices.

Using multiple gateways increases the upload bandwidth to CTERA Portal, assuming the WAN bandwidth is sufficient. Once all the data is synced up to the portal, via the gateways, the environment is protected with full disaster recovery in place. CTERA recommends that no cutover to CTERA Gateway is done until a full copy of the data is available in CTERA Portal.

There is no data size limit per gateway as the Caching gateway clears space as required as it syncs up to the portal.

For the example used in this section, each gateway is able to sync up information to the portal at a rate of 25MBps. Actual results may vary based on file sizes, latency and packet loss. Knowing the sync rate available at the site for the migration is important to determine the number of ingestion gateways required for the optimal migration.

Note: The gateway adds Everyone and Creator owner permissions by default. In order to allow ACL permissions to be fully preserved, without any deviations from the original permission set, do the following before copying the data to the gateway:

i Create a folder, one level below the cloud drive level. On this folder you can disable the inheritance while allowing inheritance in the level below.

ii After setting the permissions for the top level, run Robocopy as described below.

This document describes this procedure using Robocopy. There are also commercial tools available, such as GS RichCopy 360 Enterprise from GuruSquad.

To migrate the data:

1 Stage A: Initial migration pass.

a Enable ACL emulation on shares.

Note: ACL emulation must be enabled on shares before starting the actual migration.

b Create a designated user as an Owner of the Cloud Folders and data. CTERA recommends creating the owner as a Service Account with administrator privileges and not a real user, and that the account is a local account to the portal and not an Active Directory domain account. Once the data is uploaded to the CTERA Portal there is an owner for the data who can get elevated rights.

Note: The Owner must be a read/write administrator.

c If cloud backups for the CTERA gateway are defined, disable these during the migration and re-enable them after the migration.

d Deploy all the CTERA gateways that are to be used as the ingestion gateways.

For each gateway run Robocopy on the data from original file server to copy to the desired CTERA gateway destination.

Robocopy {source} {destinationShare} /e /v /np /copyall /xo /fft /purge /MT:64 /r:0 /w:0 /tee /log+:C:\Temp\copyLog.txt

where:

/e – Copies subfolders including empty folders. Note that if the destination folder exists, the destination folder security settings are not overwritten.

/v – Produces verbose output and shows all skipped files.

/np – The number of folders or files copied so far is not displayed.

/copyall – Copies all file information: data, attributes, timestamps, ACLs, owner and auditing information. copyall is equivalent to /copy:DATSOU.

/xo – Excludes older files.

/fft – Assumes FAT file times (two-second precision).

/purge – Deletes destination folders and files that no longer exist in the source. Note that if the destination folder exists, the destination folder security settings are not overwritten.

/MT:64 – Creates multi-threaded copies with 64 threads.

/r:0 – When a copy fails, does not retry.

/w:0 – Does not wait after a failed copy.

/tee – Writes the status output to the console window, as well as to the log file.

/log+:C:\Temp\copyLog.txt – Appends the status output to the existing log file. Creates the log file if it does not exist.

Note: If copies fail with an error 5: Access is denied message, check the permissions for the files that failed and then rerun the Robocopy command. If the problem persists, contact CTERA support.

For more information, see http://technet.microsoft.com/en-us/library/cc733145.aspx.

e Wait until syncing the files to the Portal completes. View the sync status in the Cloud Drive > Cloud Drive in the gateway user interface, to determine when the syncing has completed.

2 Stage B: Final migration pass.

a Configure the source file server shares to read only mode so no further changes can occur at the source.

b For each ingestion gateway run Robocopy on the delta of the data from original source to copy to the desired location on CTERA Gateway.

Robocopy {source} {destinationShare} /e /v /np /copyall /xo /fft /purge /MT:15 /MAXLAD: <less_than_the_number_of_days_since_last_delta> /r:0 /w:0 /tee /log+:C:\Temp\copyLog.txt

where:

/maxlad:<N> – Specifies the maximum last access date (excludes files unused since N).

3 Stage C: Cutover.

Direct all the users from the original file server to the CTERA gateway.

Users can now access and work on the CTERA Gateway.

Migrating On Share By Share Basis

If you cannot migrate all the original file server in one go, you can migrate each share separately. In this case, CTERA recommends that the shares are migrated in chunks based on the gateway size, bandwidth, and latency.

The procedure is the same as described in Migrating Using One or More Ingestion Gateway, with the following differences:

Option A

Use ingestion gateways that are not used as production gateway to perform the Robocopy. Ingestion of data can cause poor user experience for users accessing the gateway for production.

Option B

Migrate the data using Robocopy directly into the production gateway.

Use Robocopy with the following flag in order to throttle the migration speed:

Robocopy {source} {destinationShare} /e /v /np /copyall /xo /fft /purge /IPG:8000 /r:0 /w:0 /tee /log+:C:\Temp\copyLog.txt

where:

/IPG:8000 – Specifies the inter-packet gap to free bandwidth on slow lines.

By throttling the migration speed, the impact on users that are already using the gateway in production is minimized

Cutover

Direct all the users from the copied share to the CTERA gateway.

Users can now access and work on the share on the CTERA Gateway.

Estimated Migration Duration

The following estimations provide a starting point to estimate how long a migration takes after factoring for different environments. The estimations are based on the following assumptions:

Source file server contains 120TB, with average file size of 1MB. The data is split equally between three shares.

Three CTERA gateways are used in parallel, capable of 25MBps per gateway.

WAN link upstream is 1 Gbps.

Robocopy local performance to each ingestion gateway is 95MBps.

Stage A

Initial Robocopy process works at 95MBps per gateway and completes in approximately 4.9 days.

In parallel to Robocopy, the gateways will sync to cloud at 25MBps each, this initial sync will complete in approximately 18.5 days.

Stage A total: 18.5 days

Stage B

CTERA recommends executing this stage over a weekend, during which time users will continue to have read only access but no write access.

5 Hours for the Robocopy delta local copy to finish (assuming 5% change) per each of the three CTERA gateways in parallel.

In parallel to the delta copy, 5% of the data has changed (6TB), which takes approximately 23 hours to sync to the portal.

Stage B total: 23 hours

Stage C

Cutover after 23 hours of limited access.