UC Migration Data Management
Problems data management during UC migrations is one of the most common causes for issues and delays as is typically a manual process with no standardized process.
Setting the appropriate customer expectation for data migration can greatly assist, if understood early in the process. Some of the common questions i ask are
What level of data needs to be migrated? All data or only user data?
Who is responsible for validation?
What is that acceptable change freeze?
AND avoiding any mention of like for like!
Determining End-User Requirements
Migration efforts should include activities to effectively collect and provision not only the legacy user data for line appearance and accompanying settings, but also incorporate a process for the collection and integration of newly available features into the Unified Communications platform
Existing on-premise
Careful analysis of the existing configuration can assist with understanding how the current system functions and provides many inputs into the Low-Level Design (LLD), that would otherwise need to be collected from the customer. These are some items typically extracted and analysed
Device Pools – Site layout
Locations, Regions – Existing bandwidth and codec use
Trunk and Gateways – Trunking requirements
Translation and Route Patterns - Dial plan requirements
Device Mobility – IP Address Management (IPAM)
Media Services – MOH sources
Service and Enterprise Parameters exceptions
SIP profiles – interoperability
CTI objects – 3rd party systems
Extensive user and device data can be extracted from an existing on-premise solution that can result in a migration to a new system without users even being aware of the change. Although this requires extensive data extraction, this can be assisted with the use of Export Transform Load (ETL) tooling.
Data can be extracted using a variety of techniques:
o Bulk Administration Tool (BAT) - provides an easy to use bulk data export, that provides most user, device and feature data in an CSV format
AXL - Provides an abstracted API for the CUCM configuration data model that is well defined and easy to consume, that provides access to most user, device and feature configuration.
For large systems, this data extraction process can take a long time especially for older CUCM versions that have conservative rate limits.
SQL - Provides low level access to the CUCM database that allow for extraction of all configuration information
This interface requires detailed knowledge of the CUCM data model but provides the most comprehensive data extraction with the added benefit of allowing for fast data extraction
3rd Party
The level of data that can be extracted from 3rd party systems varies greatly between vendors and models and which typically require special domain knowledge to extract and transform. If possible request configuration and usage reports from the current maintainer of the system. These systems often have a significant amount of redundant configuration that requires careful verification.
Often TDM station data information is intermingled with years of legacy data that will require pruning. TDM system expertise within the team for accessing extracting and formatting TDM system data can assist greatly.
Greenfields
For new sites or where there are no existing systems, data must be collected from users. This can have a varying degree of detail depending on if new numbers are being allocated where only optional features and services need to be collected or if migrating from an existing system with no data extraction.
This data collection process is most effective if driven by the end customer to collect this data, as they are familiar with the users, if this is not possible on-site detailing is required
Data Validation
Validation of data is crucial, as unless the original system was well maintained there is likely to be a significant amount of redundant configuration that is no longer required. Identification of users, devices and features that are no longer required requires careful analysis.
Call Data Records (CDR)
Usage data can be analysed to determine if any phones and features have had any activity such as:
Devices
Extension mobility profiles (UDP)
Hunt Groups
Pickup Groups
Single Number Reach (SNR)
Once this usage data has been collected it can then be used to correlate inactive users and features. This analysis should also be extended to features such as hunt groups and pickup groups, to remove inactive entries and to ensure no provisioning errors occur when provisioning the new system.
The analysis of usage data prior to a migration can greatly assist in the data cleansing activities as users and features with no activity, can be highlighted.
Real Time Status (Risport)
Real time data can be extracted to find the registration status of devices on the existing system(s) to identify unused phones. This needs to be used selectively as devices like soft and mobile clients (Jabber) are not permanently registered, Risport can be used to also extract data if a phone has registered recently (since last CUCM service restart) that can assist with identification of Jabber clients).
Risport also provides information of registered IP address, that can assist with identification of devices that are have been moved to another site. If device mobility is configured this is easy to identify, otherwise an export from the IP Address Management (IPAM) will need to be used to compare and identify exceptions. Additionally, Risport provides the active firmware load that can help identify firmware upgrade paths if required.
Extension Mobility
Historical extension mobility information is stored in the CUCM database which can be used to find what device(s) a user is currently logged into, which can then be used to log users in automatically post migration. In addition, it can be used to find users that have never logged in.
Active Directory
User configuration can be compared with Active Directory (AD) or other LDAP systems to find discrepancies such as phone numbers, locations and other metadata and highlight these exceptions that will likely require additional attention
Deduplication
There is often a significant duplication of configuration templates within an existing system. This is most common for Phone Button Templates (PBT) and Soft Key Templates (SKT). Deduplication of these templates and normalisation of naming conventions makes pre-work for the migration much simpler and easier to support ongoing.
Data Transformation and Loading
Once the data has been verified and cleansed, it must be transformed into the format required by the domain manager in use.
Usage of bulk loaders requires experience and is dependent on the accuracy of the data being loaded, as configuration exceptions or missing dependencies can cause errors and failures for data loading and cause expensive rework for deployment engineers.
The use of tooling can assist with this data loading process as dependencies can be checked and data formats and naming standards enforced before data loading.
Challenges
These are some common challenges experienced during this phase:
Existing System Access
If the existing system is managed by another partner, sometimes there is an unwillingness to provide system access due to the privileged level of system access required, as this has the ability to make configuration changes that could impact their current Service Level Agreements (SLA). Another challenge that arises is where the partner managing the current system considers their configuration as their intellectual property, which can apply to system and design documentation and therefore also system access.
In this scenario, it is important that this is identified early in the project, as negotiations for system access and documentation release can result in delays. This requirement is usually placed upon the customer to facilitate.
Requesting BAT or other system static exports can be used in these scenarios.
Data Freezes
The use of data freezes are to allow for the data extraction, validation and loading tasks to occur, where no configuration changes are made to the existing system. It is agreed that during data freeze periods, if changes are made they will not be reflected in the new system. Determining the appropriate duration depends on a variety of factors such as the size of the system, average number of changes and customer expectations.
Tooling can be used to provide configuration delta reporting during this period, where once the new system has been configured, the delta can be applied with attention to user facing features such as call forwarding. Use of CRM reports or manual configuration change registers can also be used during this period.
Validation Accountability
Another challenge that often comes up is who is responsible for data validation. This should be identified at the beginning of the project and if required, factored into the migration costs, as if the partner is required to take on this responsibility, on-site detailing is typically required.
Benefits of Automating Data Management
Extraction and validation of user information to eliminate error prone manual data entry activities
Validation and cleansing of existing telephony data (hunt groups/pickup groups/shared lines/etc.)
Standardise the migration process, whilst retain flexibility to meet end customer requirements
Compare migration data against other sources such as active directory Active Directory or HR systems
Minimise and configuration exceptions that would impact provisioning and deployment
Accurately capture information for billing activities including billing or configuration exceptions
