A default set of file metadata, such as creation date/time, last modified time, filename and file size, are transported over on all Shuttle jobs where supported by the target cloud provider.
However, many applications, such as content management systems, further apply custom metadata to files. These may relate to review cycles for the document, items such as customers or inventory that are connected to the document, or groups/departments that the document relates to. Transferring this metadata becomes essential to maintaining the files’ utility if they are moved to a new platform.
This intro provides an overview of metadata concepts that will prepare you for the how-to articles on metadata configuration that follow in the knowledgebase.
Metadata is currently available for Box targeted jobs. Contact your Shuttle sales rep for details on adding this option to your account, or for information on metadata for sources > targets other than Windows > Box.
Metadata Mapping – Fundamental Concepts
Metadata mapping is fundamentally like user account mapping: metadata templates and metadata fields on the source are respectively mapped to metadata templates and fields on the target. Example: You have a file invoice.pdf, and it contains the following metadata:
- Customer ID: 12345
- Invoice Status: Paid
- Sales ID: 7890
To move the metadata to a new target filesystem you will do the following:
- Export the metadata for the invoice.pdf file on the source to a .csv file. The .csv file’s column headers become the metadata source template.
- Create a template of the metadata structure on the target, and match up the metadata fields between the source and target.
The example below covers metadata mapping for just one file, invoice.pdf, but in practice your metadata mapping will process multiple files.
Example of How Metadata is Transferred for One File
Create Metadata Data and Template on the Source
External to Shuttle, you will need to generate a .csv file that contains metadata for your invoice.pdf file:
|SOURCE PATH||CUSTOMER ID||INVOICE STATUS||SALES ID|
Create Metadata Template on the Target
On the target, you will need to build a template to which you can match up the source metadata. Your target template might have these fields:
|CUSTOMER NO.||INVOICE STATUS||SALES ID||SALES OFFICE ID||FUTURE USE FIELD|
You’ll notice a few things:
- The Source Path field is not in the target template on the cloud provider. Source Path just defines the file in the source template, so you don’t need to add that to the target template.
- The metadata fields don’t need to have the same names on the source and target template. You can map Customer ID on the source to Customer No. on the target, for example. Note that data types (text, number, date), which are specified in the target template you build at the cloud provider, do need to match from source to target. Furthermore, the format of the date field on the source must be converted to the format used at the target. You specify the date format in the mapping spreadsheet and Shuttle will convert dates automatically. See Account Mapping – Metadata Job Setup for additional details on how this works.
- You can have additional fields in the target template that are not present in the source template. They will just remain blank for the files that are transferred over. In this example, the field ‘Future Use Field’ is an additional field on the target that will remain blank.
Map Metadata Fields on Source to Metadata Fields on Target
Lastly, you need to match fields on the source to fields on the target in the mapping spreadsheet:
|SOURCE FIELD||TARGET FIELD|
|Customer ID||Customer No.|
|Invoice Status||Invoice Status|
|Sales ID||Sales ID|
|Sales ID||Sales Office ID|
You can map a single field on the source to multiple fields on the target. In the example above, ‘Sales ID’ on the source is being copied to two different metadata fields on the target, ‘Sales ID’ and ‘Sales Office ID.’
Every metadata field on the source must be mapped to a field on the target, or the spreadsheet validation will fail, but as previously mentioned, you can have unmapped fields on the target. And while all columns in your metadata .csv file will be reported to the “Metadata CSV to Templates” tab, you can delete these rows from the spreadsheet, they will be ignored, and the map will also validate.
You do not need to have a metadata listing in the .csv for every file selected for transfer. Files that are not listed in the metadata .csv will transfer over without any metadata.