The pressure from the specialist departments to replace cumbersome relational databases with more flexible NoSQL and cloud databases is growing. A simple migration scheme helps with the transition.
“Never change a running database.” This golden rule of database administration is becoming increasingly obsolete. New business models require new applications, and they need agile, highly scalable, and available databases that can be seamlessly integrated into cloud technologies with containers and edge computing.
An e-commerce portal can hardly be operated with a purely relational database management system ( DBMS ). The rigid table structure of the relational SQL databases and the transaction-oriented fixation prevents the necessary flexibility. The increasing use of microservices, with which SQL databases are incompatible, speaks for the use of modern NoSQL databases. Not to mention the many automation features that simplify installation and operation.
This puts the unloved topic of migration on the agenda for many companies and their IT departments. You have two modernization options: NoSQL and cloud databases. There is an inevitable confusion of terms when it comes to cloud databases. On the one hand, this describes a new database type typically offered and hosted by hyperscalers. On the other hand, it describes a general provision model for databases used in cloud services (DBaaS).
Cloud databases can scale out and support big data and microservices like NoSQL databases. However, cloud-agnostic is not part of the repertoire since they are usually tied to a specific provider.
The Start Of The Transition
On the other hand, NoSQL databases can, in principle, run independently in any private or public cloud and demonstrate their advantages over conventional relational databases there. However, the prerequisite for this is fulfilling automation level 5 (full automation), as defined in the operator framework as the highest level.
The requirements listed there include the ability for unidirectional and bidirectional replication across multiple data centers or regions. This means that the database is cloud-agnostic and infrastructure-agnostic, and thus independent of cloud platforms such as AWS, Google Cloud, or Microsoft Azure and all other deployment models.
The hyperscalers mentioned facilitating the migration of NoSQL databases to their clouds with standardized processes, migration tools, and detailed documentation. The preconfigured networks and firewalls of a public cloud also eliminate the conflicts that often occur with private clouds.
Irrespective of this, a particular sequence of migration steps in three steps has proven to be expedient in practice:
- The migration of the users
- The data migration
- The migration of the indices
The migration of users is a preliminary first step in which the access authorizations and authorization levels of the users for the new database are determined. This ensures that no unauthorized person has access to database functions and databases.
As a rule, a distinction is made between administrators (Admin), users who are authorized to make changes (Edit), and those who only have read access (View). The transfer of authorizations to the new database can be automated using a script.
Data Transfer: Sometimes Easy, Sometimes Difficult
The second, most critical, and usually most time-consuming step is migrating the entire database from the old SQL to the new NoSQL database, during which nothing can go wrong. Database managers can choose between two different methods: offline and online migration.
The option for online migration is usually used. This process is speedy and convenient for a database with cross-data center replication (XDCR) capability because the database can be replicated asynchronously into the new database. A process that is completed in just a few seconds at the push of a button and in which the data stock in the old database remains consistent.
With offline data migration, on the other hand, the data has to be extracted manually from the old database and then loaded into the new database, for example, in the form of a CSV file. During this export/import period, which can sometimes last more than an hour, the database is in downtime mode, i.e., unavailable. A connected application, such as a production control program, cannot be used, and production is at a standstill. Therefore, offline migration only makes sense in a few cases, for example, when internal policies prohibit opening the firewalls for data export to the cloud, which is necessary for online migration.
With the data transfer, the essential migration stage is completed. Finally, the indices are migrated. This also applies to the particular case of N1QL index migration. N1QL is an exclusive query language that allows JSON datasets to be queried using SQL statements, further leveraging widespread SQL know-how. By using this migration scheme, changing the database management system loses its horror. In particular, the automation functions and the options for migration during operation shorten and simplify the switch to a modern NoSQL database enormously.
Also Read: NoSQL Database: Intercepting Failures In Data Centers And Cloud Networks
