How to migrate data painlessly when switching to a new CTMS and eTMF

Jul 26, 2021

Recently, we discussed questions that might arise when changing your eClinical software provider. Today, we want to take a closer look at one of the most asked questions:

How can I migrate data from a legacy CTMS or eTMF?

Data migration is a delicate process with many moving parts. Without a well-established process, the CTMS go-live could be postponed, with your new provider wasting resources on getting your historical data into their database in the meantime.

Let’s talk about how Flex Databases manages the data migration process and share best practices based on our experience. Below is the typical data migration flow implemented at Flex Databases:

Let’s look at each step in more detail:

Step 1: As-is data audit

Historical data is carefully scoped and scrutinized on its structure, links, and accuracy. We always involve all stakeholders and ask them to verify their own data before the audit.

Before the audit begins, the following questions must be answered:

  • How can data be extracted from a legacy system?
  • Will this data be raw extracts and, if so, in which formats?
  • Does the system have an API?

Step 2: Source data preparation

Once we determine the scope of work and initial data structure, we identify the following field-level obstacles:

  • Potentially missing links, blanks, and extra spaces
  • Whether to convert data fields before uploading them (e.g. dates, which can vary across databases)
  • Validation and business rules in the legacy system that should be reused in the new database

Sometimes, a lack of data governance rules in the legacy system results in frustration during the data migration. Here is an example from our practice:

There are cases where a single data point would be the same across databases, but each entry would be inconsistent in terms of spelling. For example, in three databases a data point was marked as “screen-failed”“screen failed”, and “screening failure”. Thanks to our analytical tools, we managed to triage and consolidate all siloed values, creating well-organized data.

Identifying missing links in a legacy system at the early stage is a difficult task, but not impossible. With the help of automated data, the target system does not inherit weaknesses from the source system.

Step 3: Migration strategy

Using the results of data audit and preparation, and taking into account business expectations, we define a strategy for data migration and acceptance criteria. We offer two options:

One-time transferIterative transfer
The full data transfer is completed within the pre-agreed time frame while systems are in downtime.Continuous migration while both the source and target systems are running.
  1. Given that most complexities are identified during the preparation stage, we can mitigate risks and avoid unexpected issues accordingly while keeping the project within the agreed budget and schedule.
  2. We put a lot of focus on crafting the risk management policy, an essential part of any migration plan that aims to outline the issues that could happen at any stage and how they can affect expectations.
  3. Once selected, the data migration plan, including timelines, tasks, and responsible managers, is communicated to all stakeholders for transparency.

Step 4: Set up and testing

After the migration plan and strategy are set, we identify and allocate the tools and resources needed to implement each step. Core tools that we utilise belong to the following classes:

The choice of tool depends on the complexity of the migration project and processes involved. Although the old and new systems belong to the same family (CTMS), the same feature/object can vary across different CTMS.

This concerns not only the data transfer but the data transformation and remodeling process. As such, the migration tool should also support these additional activities.

As a rule of thumb, each phase of the migration plan should be covered by tests, both automated and manual. Test scripting usually involves the following aspects:

  • The test objective itself
  • The time frame of the test
  • The scope of data involved (single record, data subset, full dataset, etc.)
  • The mock data or live data used
  • Parameters for evaluating and communicating the test results

Step 5: Data transfer

Once the internal validation and testing is completed and all discrepancies resolved, we execute the live data transfer.

We ensure that all stakeholders are informed in advanced so that can plan their schedules and activities accordingly.

Step 6: Post-migration data audit

We continue to test even after the data transfer is done. The final data audit aims to ensure the completeness and accuracy of the migration and, eventually, showcase the success of the overall project.

Our approach is usually to build a final reconciliation report showing the total number of objects transferred; actual number of records transferred versus those initially targeted; and discovered inconsistencies and how they have been processed.

More about our product

Webinar recording: Digitalizing Quality Management System: How and Why? with Deloitte

Late November we’ve hosted a joint webinar with Deloitte on how you can and why you should consider digitalizing Quality Management System.

Learn more
KPI reports – get a complete overview of what’s essential at a glance

What’s usually been a whole collection of different graphs is now in one – and you can see when and if something goes wrong. 

Learn more
Anything you can do, with DMS would be better – manager & user perspective

Flex Databases DMS is done exactly right – and here’s a quick overview of manager & user processes, that get simply better with its implementation.

Learn more
WSG selected Flex Databases eTMF & CTMS to centralize and optimize study management

WSG selected Flex Databases eTMF & CTMS to centralize and optimize study management.

Learn more

Have we picked your interest?

Tell us more about yourself and we will organize a tailored live demo to show how you can power up your clinical trials processes with Flex Databases.