73 - Data migration? Avoid these 5 pitfalls

EPM implementations and data migration: avoid these 5pitfalls12Minutes estimated reading time

When implementing a new EPM system, data migration is often a challenge. Not infrequently, data migration causes delays or unforeseen additional efforts. In this article, I will take you through the 5 most common pitfalls. Including recommendations to avoid them.

Pitfall 1: Underestimation
Pitfall 2
: Viewing data migration as a purely "technical" matter
Pitfall 3
: Mixing goals and activities
Pitfall 4
: One-time exercise
Pitfall 5
: Unrealistic or undesirably large scope

Pitfall 1: Underestimation

When implementing a new EPM tool, the initial focus is not on the necessary data migration. Both the client and the consulting partner have other priorities. Such as resolving pain points in the current reporting process and implementing new functionalities.

Consultants focus primarily on understanding the business, processes, reporting requirements, delivering data streams and integrations to be set up. They are responsible for the initial integration and usually view data migration as the client's responsibility.

But that customer's main focus is on getting to know and understand the new tool. In short, data migration doesn't get the attention it deserves. Consequence? Underestimation.

Underestimating the amount of work and lead time

Reconciling and (re)loading data always takes more time than you think. Moreover, stakeholders often do not start detailing the required data quality and quantity until late in the project.

Underestimating the burden on those involved

(Co)working on the data migration is a secondary task for designated staff and consultants. Not only does the implementation itself require time and attention. Regular reporting must also continue as usual.

So on top of that comes another task: freeing up time for data migration and reconciliation in a new system whose ins and outs employees do not yet fully understand.

Underestimation of different data needs

What is sometimes forgotten: you also need data during implementation. Think of data for testing by the consultants (Unit test) and by the users (UAT) and data for training purposes. Must that data be available, of course.

Preventing underestimation

My recommendation to avoid underestimation? Schedule migration work already during project planning. Discuss and document requirements as early as possible. And preferably assign someone directly responsible for data management and scheduling the delivery of the various data during the project.

The result will be a more effective, efficient data migration and a higher quality final delivery.

Pitfall 2: Viewing data migration as a purely "technical" matter

Data migration is usually more than just "mapping, loading and comparing/reconciling. Indeed, in many cases, the migrated data is not directly comparable to the original data.

Inconsistent sources

Inconsistent sources are a known cause. Such as data that has been modified by manual corrections in reporting. Or a changed data structure that has not been implemented consistently. Sometimes bases for derived data have changed over time. That, too, can lead to inconsistent sources.

Changes due to design

Another possible cause of data conflicts lies in the new design. Consider changed bases for calculations, for example for the method of translation or elimination. We also often see clients taking advantage of an implementation to revise the Chart of Accounts.

Interpretation of differences

During the migration, you have to constantly make substantive trade-offs:

  • Is the data comparable to what was previously reported?
  • Can the differences be understood and explained?
  • Do the changes impact derived values such as KPIs, cash flow or reporting?
  • Are additional measures needed to match the data with the past?

In short, migration and acceptance of data is not merely a technical matter, but also a functional one. Especially when it comes to the interpretation and assessment of the data. Keep this in mind when estimating the amount of work and expertise required.

Pitfall 3: Mixing goals and activities

As I wrote, there are different data needs throughout the project. Hence the advice to start data migration in the development environment from the very beginning so that you have test data at hand.

However, it is important not to let this test data be part of the data migration itself. You want to prevent the data migration from hindering further development. Migration in the test phase of the project has two purposes:

  1. Testing out migration (strategy and technique)
  • Is there enough detail in the source files?
  • Does the source satisfy qualitatively?
  • Does the transformation during the migration deliver the right outcome?
  1. Providing test data needs for the project, both for builders and users/testers.
  • Are the dimensions and COA set up and configured correctly?
  • Data can be used to test Consolidation, Aggregation, Translation and Elimination in the system.
  • Data can be used as a source for setting up additional calculations and derived values.

Data migrated in this phase should not interfere with application testing and modification. Structural change may require you to delete existing data. Testers can make manual additions to test things. But changes to the application may lead to unreliable data.

Separate work

The nature of the work of development and testing soon comes into conflict with the nature of work of data migration and reconciliation. Hence the advice to separate the more time-consuming and structured migration work from ongoing development. After all, migration limits the development environment. This prevents the migration from continuously shooting at a moving target.

Pitfall 4: One-time exercise

Migration is still sometimes seen as a one-time task. But during development, the need often arises to make changes to the system. These changes can cause the migrated data to become non-compliant.

For example, because the solution does not meet the desired functionality or performance. Or because new needs arise during the project that must be met.

In addition, there is sometimes a need for data enrichment to supplement the migration of data from the old source system. This may be necessary to meet the new requirements arising from the design of the new solution or to correct the aforementioned inconsistencies in the source data.

So keep in mind that the migration may have to be redone in modified form.

Recommendations

  • Create repeatable and documented processes.
  • Provide well-documented versions of source files.
  • Avoid manual corrections in the system.
  • Document previously selected differences.
  • If possible, create separate sources/processes to load or book historical reconciliation corrections. In addition, make the various data flows transparent in the new solution, for example via a separate dimension. This will support the reconciliation process and also facilitate a possible audit afterwards.

Think of data migration as a process to be designed during the project with the apotheosis being the final migration for the go-live. Depending on the size of the project, the design of the migration can be recorded in a separate document. This allows the customer to get to know the (new) migration platform already in the initial phase. In many projects, the tool used for migration is the same one used for setting up integrations.

Pitfall 5: Unrealistic or undesirably large scope

If you ask an end user what data needs to be migrated, an undesirably large scope often follows. The effort - and thus the cost - of arriving at the migrated data soon becomes disproportionate to the benefits and ultimate use.

Also realize that the further you go back into the past, the more discrepancies and imperfections appear in the source data.

Paradoxically, on the one hand, there is sometimes a desire to bring historical "ballast" in the form of old accounts and entities into the new solution. While at the same time there is the desire to clean up as much as possible. Well...

Avoiding unrealistic scope

You can often avoid an unrealistic scope by giving the end user and/or stakeholder insight into the additional costs and efforts.

Also effective: making the end user feel the pain of the extra effort by making him/her commit extra effort for all data migration. And that with every scope expansion.

In any case, what you can do is examine the quality of the source data in the initial phase. Based on that, you will get a more realistic picture of the data migration effort.

In addition, it is good to test the actual use of "comparable data" against daily practice. Often it turns out that the really needed comparable figures, for example for financial statements, do not come from the EPM system. For common longer-term comparative figures, migrating main lines is sufficient.

To comply with the required legal retention period, a good archiving strategy is usually more effective.

Minimal data migration

Sometimes it is better to really keep data migration to a minimum. Namely, when the sources are too disjointed and unreliable. For example, if Excel is the basis, with many manual corrections and an unstructured, ever-changing setup. In such a situation, it is better to focus on a good opening balance sheet to get the new system off to a good start.

In conclusion

This article does not claim to be complete, scientific or the only truth. Its main purpose is to inspire you as you get started with data migration within an EPM project. Every EPM project is different, every customer is different. But above all, think carefully about the right approach and be alert to the 5 pitfalls mentioned.

Want to know more or ask questions?

Want to exchange thoughts on this topic without obligation or have experiences you'd like to share? Feel free to send me a personal message.

Do you have more specific questions about the various tools or do you need concrete support? Then of course you can also contact us. Especially when it comes to Oracle and OneStream. At Bart & Partners we have purely experienced people on the team. So chances are that we can answer your specific question. And if not, we'll just say so.

Smart Simple, Solid 😊

About the author Martijn is an experienced EPM expert who has mastered all aspects of the EPM lifecycle. From project implementation to management and from change management to data migration.