Migration is one of those words that conjure up fear in the hearts of IT professionals. Migration is a word associated with hardship and danger when referring to the migration of people. Data migration is no different especially when Product Lifecycle Management (PLM) tools are involved. The best description for PLM data migration I have heard was in a meeting with Dell. One of their IT resources described it as "changing out the airplane engine while the plane is still flying without crashing the airplane. I think this is a pretty apt description. Migration can be an afterthought in a lot of PLM implementation projects but it definitely can dictate the success or failure of PLM and if done improperly it can result in a dramatic loss of altitude ending with an equally dramatic deceleration event as you hit the ground. Zero Wait-State specializes in data migration and we have spent the last decade migrating companies both large and small off of a variety of PDM and PLM platforms. This blog will highlight some of the perils and pitfalls around PLM data migration and best practices that can keep you flying high.
Garbage in Garbage Out
One of the first mistakes companies make when embarking upon a data migration project is assuming their current data structure is worthy of migration. Over the year companies accumulate a large amount of information and not all of it is critical to the company going forward and not all of it is in a condition that will make it useful in the new system. This would be a good time to take a hard look at your legacy data and determine what really needs to come over into the new system. Older product information may no longer be relevant and some information is not structured in a way to be useful in a PLM system. Obviously when you are moving from a PDM or PLM tool into a new PLM tool you are able to capture useful metadata like attributes but many companies migrate from system disks to PLM which can be much more challenging. Time needs to be spent evaluating current data and how relevant it is for future product development needs. Also if data is corrupted or sub optimal you should think hard before polluting the new system with the data.
2 Gallons of Water in a 1 Gallon Container
This is more of an issue for companies who are downsizing or as they like to say "rightsizing". Companies that elect to move from a more sophisticated PLM to a more affordable solution or an easier to use system can run into issues where they have more information to move than the new system can accommodate. Examples of this might be moving from Agile Advantage to Arena or from Intralink or PDMLink to Enterprise PDM(EPDM). The source system might support more data type or attributes than the new system can deal with so you need to take this into account when preparing the information to move over. More appropriately you need to consider this before you make the decision to move. If any of the information is critical you may be in for an unpleasant surprise. Workflows, Change history, file attachments, and access control logic are all things that may not be accommodated when moving from one vendor to another.
What a Tangled Web we Weave
It is not unusual for a company to want to migrate multiple PDM's or data sources to a single environment. While I certainly see the value in doing something like this it adds significant complexity to a data migration effort. Issues like naming conflicts and data integrity can severely hinder a migration involving multiple data sources. Coordination between the sets of information is essential and in depth analysis up front will save major issues from appearing after the migration is complete. Most PLD and PDM systems will not allow non-unique names so it is best to work out these issues prior to migrating into the target system. The more consolidation you can do prior to migration the less effort you will have to expend during and after the migration to the target system.
Volume, Volume, Volume
Some of the projects we have worked on involve massive amounts of data. I am sure you are aware of how much information even a small company can accumulate over time. This creates some challenges when it comes to migration. The first challenge is being able to convert all of the information in a timely manner. One of the keys to being able to change out the airplane engine in mid air is being able to do it at a time when no new information is being added to the source system. Large data sets can take several days to process so this can be a problem. There are two ways to resolve this issue. One is reducing the amount of data through purging or deletion the other is to migrate in phases. Phased migration makes a lot of sense for most companies because it allows you to break up the activity into manageable pieces. It also allows for validation and testing as you go. The idea is that you take periodic snapshots of the data you are moving and test the data to validate the process. Once the data has been validated you can do a final migration of just the delta between when you took the snapshot and the most current version of the source data.
History is Best Forgotten
There is a famous quote from the philosopher George Santayana saying, "Those who cannot remember the past are doomed to repeat it." George obviously has never been involved in data migration projects. History from PDM and PLM systems adds a degree of complexity to a migration that can be prohibitive. Extracting and capturing history from legacy PLM and PDM systems is very challenging. Ensuring that this information gets accurately transmitted into a target system can be even more problematic depending upon the sophistication of the import utilities. Using a "latest only" approach dramatically lessens the complexity of the migration and most companies discover that their history wasn't that useful. If it is absolutely necessary to preserve the history some companies will elect to keep their legacy system around or virtualize it and let people access it on an "as-needed" basis. It turns out for most companies that the need is surprising less than they expected. This also addresses some of the issues I mentioned above related to "garbage in garbage out" and "volume, volume, volume".
Be Prepared
The famous Boy Scout motto applies to many things but especially to PLM data migration. Preparation starts with hardware. It is a big mistake to under provision servers for this type of effort particularly if you are planning to move a lot of data. The other thing to be prepared with is a good set of data analysis and cleaning tools. As we discussed above understanding the condition of your data and being able to automate some of the cleanup can be very impactful. In most cases particularly with metadata there are scripts and applications to detect naming conflicts and to validate data transmission on the backend. It is important that you either have these capabilities or work with someone who does. You need to walk through the analysis process and understand the condition of your data so you are not caught off guard by the level of cleanup you will need on the back end of the migration. Ideally you should do as much clean-up as possible prior to moving information into the target system. It is infinitely easier to clean up information prior to migrating into a new PLM tool.
Test, Validate, Rinse Repeat
It is almost impossible to test too much during a data migration. We recommend two test passes minimum which should allow you to go into a test system and thoroughly sample the data before eventually moving to production. A well considered test plan should be included for the project either developed internally or with a partner's assistance. Ad hoc testing is better than nothing but there is a good possibility that you could miss something. Generic test plans are better than ad hoc testing but again without tailoring something specific for your environment you may find out the hard way that something is missing or wrong. If your company is required to validate per FDA or other regulations you can combine some of this effort into testing and potentially compress your testing cycle. It is best to get testing addressed up front before the project gets too far down the road. The temptation to truncate testing is great so you must resist.
The End of the road
Most of these ideas are common sense but surprisingly often are disregarded in data migration efforts. Too many times the migration project is just bolted on to a PLM implementation as an afterthought and in the rush to get the new system into production the migration discipline is compromised. It is important to recognize that PLM tools are just vessels that hold the most important asset a company has; their intellectual property. It is far more important to ensure that information is transferred cleanly and accurately than it is to stand up a new PLM system. Plane crashes are tragic and obviously anything that happens with migration pales in comparison with the loss of life experienced but in the context of business nothing can be more crippling than losing product information. This data is the lifeblood of a product development company so safeguarding it should be one of the highest priorities a company has.