Migrating dates into Drupal

Today we will learn how to migrate dates into Drupal. Depending on your field type and configuration, there are various possible combinations. You can store a single date or a date range. You can store only the date component or also include the time. You might have timezones to take into account. Importing the node creation date requires a slightly different configuration. In addition to the examples, a list of things to consider when migrating dates is also presented.

Getting the code

You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is `UD date` whose machine name is `ud_migrations_date`. The migration to execute is `udm_date`. Notice that this migration writes to a content type called `UD Date` and to three fields: `field_ud_date`, `field_ud_date_range`, and `field_ud_datetime`. This content type and fields will be created when the module is installed. They will also be removed when the module is uninstalled. The module itself depends on the following modules provided by Drupal core: `datetime`, `datetime_range`, and `migrate`.

Note: Configuration placed in a module’s `config/install` directory will be copied to Drupal’s active configuration. And if those files have a `dependencies/enforced/module` key, the configuration will be removed when the listed modules are uninstalled. That is how the content type and fields are automatically created.

PHP date format characters

To migrate dates, you need to be familiar with the format characters of the `date` PHP function. Basically, you need to find a pattern that matches the date format you need to migrate to and from. For example, `January 1, 2019` is described by the `F j, Y` pattern.

As mentioned in the previous post, you need to pay close attention to how you create the pattern. Upper and lowercase letters represent different things like `Y` and `y` for the year with four-digits versus two-digits, respectively. Some date components have subtle variations like `d` and `j` for the day with or without leading zeros, respectively. Also, take into account white spaces and date component separators. If you need to include a literal letter like `T` it has to be escaped with `\T`. If the pattern is wrong, an error will be raised, and the migration will fail.

Date format conversions

For date conversions, you use the `format_date` plugin. You specify a `from_format` based on your source and a `to_format` based on what Drupal expects. In both cases, you will use the PHP date function’s format characters to assemble the required patterns. Optionally, you can define the `from_timezone` and `to_timezone` configurations if conversions are needed. Just like any other migration, you need to understand your source format. The following code snippet shows the source and destination sections:

source:
  plugin: embedded_data
  data_rows:
    - unique_id: 1
      node_title: 'Date example 1'
      node_creation_date: 'January 1, 2019 19:15:30'
      src_date: '2019/12/1'
      src_date_end: '2019/12/31'
      src_datetime: '2019/12/24 19:15:30'
destination:
  plugin: 'entity:node'
  default_bundle: ud_date

Node creation time migration

The node creation time is migrated using the `created` entity property. The source column that contains the data is `node_creation_date`. An example value is `January 1, 2019 19:15:30`. Drupal expects a UNIX timestamp like `1546370130`. The following snippet shows how to do the transformation:

created:
  plugin: format_date
  source: node_creation_date
  from_format: 'F j, Y H:i:s'
  to_format: 'U'
  from_timezone: 'UTC'
  to_timezone: 'UTC'

Following the documentation, `F j, Y H:i:s` is the `from_format` and `U` is the `to_format`. In the example, it is assumed that the source is provided in `UTC`. UNIX timestamps are expressed in `UTC` as well. Therefore, the `from_timezone` and `to_timezone` are both set to that value. Even though they are the same, it is important to specify both configurations keys. Otherwise, the from timezone might be picked from your server’s configuration. Refer to the article on user migrations for more details on how to migrate when UNIX timestamps are expected.

Date only migration

The Date module provided by core offers two storage options. You can store the date only, or you can choose to store the date and time. First, let’s consider a date only field. The source column that contains the data is `src_date`. An example value is `2019/12/1’`. Drupal expects date only fields to store data in `Y-m-d` format like `2019-12-01`. No timezones are involved in migrating this field. The following snippet shows how to do the transformation.

field_ud_date/value:
  plugin: format_date
  source: src_date
  from_format: 'Y/m/j'
  to_format: 'Y-m-d'

Date range migration

The Date Range module provided by Drupal core allows you to have a start and an end date in a single field. The `src_date` and `src_date_end` source columns contain the start and end date, respectively. This migration is very similar to date only fields. The difference is that you need to import an extra subfield to store the end date. The following snippet shows how to do the transformation:

field_ud_date_range/value: '@field_ud_date/value'
field_ud_date_range/end_value:
  plugin: format_date
  source: src_date_end
  from_format: 'Y/m/j'
  to_format: 'Y-m-d'

The `value` subfield stores the start date. The source column used in the example is the same used for the `field_ud_date` field. Drupal uses the same format internally for date only and date range fields. Considering these two things, it is possible to reuse the `field_ud_date` mapping to set the start date of the `field_ud_date_range` field. To do it, you type the name of the previously mapped field in quotes (‘) and precede it with an at sign (@). Details on this syntax can be found in the blog post about the migrate process pipeline. One important detail is that when `field_ud_date` was mapped, the `value` subfield was specified: `field_ud_date/value`. Because of this, when reusing that mapping, you must also specify the subfield: `’@field_ud_date/value’`. The `end_value` subfield stores the end date. The mapping is similar to `field_ud_date` expect that the source column is `src_date_end`.

Note: The Date Range module does not come enabled by default. To be able to use it in the example, it is set as a dependency of demo migration module.

Datetime migration

A date and time field stores its value in `Y-m-d\TH:i:s` format. Note it does not include a timezone. Instead, `UTC` is assumed by default. In the example, the source column that contains the data is `src_datetime`. An example value is `2019/12/24 19:15:30`. Let’s assume that all dates are provided with a timezone value of `America/Managua`. The following snippet shows how to do the transformation:

field_ud_datetime/value:
  plugin: format_date
  source: src_datetime
  from_format: 'Y/m/j H:i:s'
  to_format: 'Y-m-d\TH:i:s'
  from_timezone: 'America/Managua'
  to_timezone: 'UTC'

If you need the timezone to be dynamic, things get a bit harder. The ‘from_timezone’ and ‘to_timezone’ settings expect a literal value. It is not possible to read a source column to set these configurations. An alternative is that your source column includes timezone information like `2019/12/24 19:15:30 -07:00`. In that case, you would need to tweak the `from_format` to include the timezone component and leave out the `from_timezone` configuration.

Things to consider

Date migrations can be tricky because they can be affected by things outside of the Migrate API. Here is a non-exhaustive list of things to consider:

• For date and time fields, the transformation might be affected by your server’s timezone if you do not manually set the `from_timezone` configuration.
• People might see the date and time according to the preferences in their user profile. That is, two users might see a different value for the same migrated field if their preferred timezones are not the same.
• For date only fields, the user might see a time depending on the format used to display them. A list of available formats can be found at `/admin/config/regional/date-time`.
• A field can be configured to be presented in a specific timezone always. This would override the site’s timezone and the user’s preferred timezone.

What did you learn in today’s blog post? Did you know that entity properties and date fields expect different destination formats? Did you know how to do timezone conversions? What challenges have you found when migrating dates and times? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.