Test to Production Metadata Migrator

Test to Production Metadata Migrator is a command line utility which helps you with synchronising your test and production IJC schemas. It's designed to migrate IJC metadata between otherwise identical environments. It allows you to prepare your entities, views and other IJC items such as Groovy scripts in your test database and migrate the changes to your production environment.
Please note that the tool does not migrate your own tables nor any data present in those tables. It only migrates IJC schema metadata, which are the records in the IJC_* tables that describe the IJC schema items.

The tool is part of Instant JChem product pack and can be obtained from Instant JChem download area . The tool is bundled as a single jar file called test2production.jar.

Please make sure that the version of the tool you use is the same as the version of the IJC schema that you will be migrating.

In order to use Test to Production Metadata Migrator tool you need to have Java 6 or later installed on your computer. You can download and install Java from the Java web site .

Please read carefully through the following sections and make sure that you fully understand how this tool works. Keep in mind that this tool is designed to modify your production environment and any mistake can potentially have severe consequences. We also strongly suggest to make a backup copy of your production environment and confirm that this backup can successfully be used to restore your production environment in case of a disaster before you attempt to use this tool!

Concepts

Identical Environments

All IJC items such as entities, their fields, relationships, datatrees are described by simple records in one of the IJC_* database tables. Each item has its own record, which is uniquely identified by ITEM_ID value (a string of hexadecimal digits). The ITEM_ID value is generated when an IJC item is created and the value never changes.

All IJC items comprise a hierarchical structure that starts with an IJC schema item. The IJC schema item is a common ancestor of all the other items within that particular IJC schema.

Test to Production Metadata Migrator tool is designed to match IJC items in two different database schemas by their IDs, find differences such as new or missing items, changed parameters of an item and update IJC items in the target database schema to be the same as IJC items in the source database schema. In order to do that the tool is given a connection string to both the source and target databases and the IJC schema ID.

The tool expects that the source and target database environments are virtually identical. More specifically this means that the databases are expected to be of the same type and version, for example both databases are Oracle 11g or both databases are MySQL 5.1 or both databases are Derby 10.2. If JChem Cartridge is installed it is expected to be installed in both databases, configured the same way and both installations be of the same version. The IJC schemas that are being synchronized are expected to be of the same version. Both databases are expected to contain the same set of your custom tables and have the same data in those tables.

Failing to meet these expectations may result in a broken IJC schema in the target database schema after using the tool.

Database Connections

Test to Production Metadata Migrator tool synchronises IJC schema items between two different database schemas. Those two database schemas are specified in the form of *.ijs files, which are the same files that Instant JChem uses for connecting to database schemas and opening IJC schemas. All that you need to do when preparing these *.ijs files before you can synchronize the IJC schemas is to open both the source and target IJC schema in Instant JChem. Instant JChem application will store the connection information to *.ijs files in your project. The same *.ijs files can then be passed on to the tool.

An *.ijs file is a simple XML file with a set of properties needed for connecting to a database schema and opening a particular IJC schema. The IJC schema to open is specified by the IJC schema item ID value. Since the tool is designed to synchronize the same IJC schema between two environments both the *.ijs files have to contain the same IJC Schema item ID.

Please note that tool will not attempt to synchronize IJC schemas with different IJC schema item IDs.

Additionally, the tool will not follow any security rules defined in any of the IJC schemas that are being synchronised. Instead it will prompt for a database username and password for both the source and target databases. The username that you use for connecting to the source database will need to have a readonly access to all IJC_* tables within that database. Similarly, the username specified for connecting to the target database will need read-write access to all IJC_* tables in the target database.

Synchronisation Details

When synchronising IJC items between the source and target database schemas the tool first determines differences between the schemas. There can be the following three types of differences.

  • Addition - There is an IJC item in the source database schema that does not exist in the target database schema. This type of difference is determined by comparing IJC item IDs between the source and target schemas. If rephrased, 'Addition' means that for an IJC item in the source schema there is no IJC item in the target schema with the same ID.

    Please note that this type of difference could either be caused by a real new IJC item created in the source schema or by an existing IJC item deleted from the target schema. This is an ambiguity that cannot automatically be resolved.

  • Removal - There is an IJC item in the target database schema that does not exist in the source database schema. Again, this is determined by comparing IJC item IDs and so more precisely this type of difference means that for an IJC item in the target schema there is no IJC item in the source schema with the same ID.

    As explained previously even this type of difference can either be caused by truly deleting an existing IJC item from the source schema or by adding a new IJC item in the target schema.

  • Modification - Both the source and target IJC schemas contain the same IJC item (determined by IDs), but the item definition is different in both schemas. This means that either the source version or the target version or both version of the IJC item have had some of their parameters changed. A very basic example of a modification is renaming an IJC item.

The following table lists all IJC item types that are currently supported and shows what types of differences are recognized and synchronized by the tool.

IJC item type

Addition

Removal

Modification

Field

yes

yes

yes

Entity

yes

yes

yes

Relationship

yes

yes

yes

Datatree

yes

yes

yes

View/Form

yes

no

yes

List

yes

no

yes

Query

yes

no

yes

Script

yes

no

yes

Performing addition, removal or modification can be disabled using --no-adds, --no-deletes and --no-updates command line flags. See Command-line Syntax for further details.

User Specific Items

The tool will by default synchronize only IJC items that are available/visible for all IJC users. However, it is also possible to synchronize IJC items that are available to a specific user only. The tool will then consider only IJC items belonging to the specified user and will ignore the others.

Please note that Fields, Entities, Relationships and Datatrees are always available to all IJC users and particular users cannot take the ownership of these IJC items. Contrary to these IJC items Views/Forms, Lists, Queries and Scripts can be owned by a user. These IJC items will not be synchronized unless you explicitly tell the tool to do so.

Using on Derby

Although, it is not considered a typical usecase it is possible to synchronize IJC schemas between two Derby databases. The procedure is exactly the same as in case of synchronizing between Oracle or MySQL databases. There is, however, one important difference. A Derby database is not designed to be used in a multi-user environment. This means that a single Derby database can only be used by a single user, process or application at any given time.
Before you use Test to Production Metadata Migrator tool to synchronise IJC schema items between two Derby databases make sure that both these database are not used by any other application. In particular make sure that these databases are not used by Instant JChem.

Command-line Syntax

This is the general command-line syntax when launching Test to Production Metadata Migrator tool.
java -jar test2production.jar [options] [source.ijs target.ijs]
The following table lists all the available options with their parameters and description. The same information can be obtained by running the tool without any command-line parameters.

Option

Parameters

Description

--apply

none

Applies all changes to the database. If this option is not present the changes are only printed for review.

--log

none

Prints extra logging output. For debugging and troubleshooting purposes only.

--users

comma separated list of users

Synchronizes only IJC schema items belonging to specified users. You should quote the list of users to make sure that the parameter is parsed correctly. For example use

--users "tom,jerry"

. If this option is missing or the list of users is empty the tool will ignore IJC items owned by users and only synchronize those IJC items that are available/visible to all users.

--version

none

Prints current version of the tool.

--no-adds

none

Skip adding new items which were created on the test platform.

--no-deletes

none

Skip deleting items which were removed from the test platform.

--no-updates

none

Skip updating of items changed in the test platform.

Examples

The typical usecase will involve running the tool several times without the --apply parameter, checking the changes detected by the tool and confirming that they can really be migrated to the target database schema. When the changes are checked and confirmed the tool can be run again with the --apply parameter, which will apply the changes and update the target database schema.

    java -jar test2production.jar ~/IJCProjects/ijc_project/.config/test.ijs ~/IJCProjects/ijc_project/.config/production.ijs
    # review detected changes printed to stdout

    java -jar test2production.jar --apply ~/IJCProjects/ijc_project/.config/test.ijs ~/IJCProjects/ijc_project/.config/production.ijs
    # double check changes performed in the target database schema printed to stdout

Importing into empty schema

  1. Create new empty schema in your database without IJC tables.

  2. Use Instant JChem to create new empty IJC schema in your target DB schema.

  3. Connect directly to target DB and change schema ID in IJC_SCHEMA table - it must be the same like schema id in the source (can be found in source .ijs file).
    It has to be changed in SCHEMA_ID and ITEM_ID columns where GENERIC_TYPE is equal to "SCHEMA".

  4. Change the schema ID in target (newly created) .ijs file - again it should be the same like the source schema id. Then just check that IJC can still connect to target schema after these changes. If so you can use test2production tool.

Limitations

The tool can not use an empty database as target schema. First IJC_* tables have to be present. And if they are, the IDs of source and target IJC schemas have to be identical. This problem may be overcome by few manual steps described in the examples .

The current version of the tool does not synchronize all IJC_* tables. In fact, it does not make much sense to synchronize some of the IJC_*

tables at all. The following list describes all IJC_* database tables and their synchronization specifics.

Database Table

Synchronized

Details

IJC_AUTHORITIES

Yes

Defines particular roles (authorities) used in IJC schema. Only new records are added to the target IJC_AUTHORITIES table. Existing records are never deleted nor updated.

IJC_CUSTOM_ITEMS

Yes

All View/Form, List, Query and Script definitions, see details here.

IJC_CUSTOM_ITEMS_SHARING

Yes

Contains sharing information for View/Form, List, Query and Script definitions.

IJC_SCHEMA

Yes

All Datatree, Entity, Field and Relationship definitions, see details here.

IJC_SCHEMA_AUTHORITIES

Yes

Determines visibility of Datatree, Entity, Field and Relationship definitions for particular roles in IJC_AUTHORITIES table. See Filtering items using roles for details.

IJC_USER

Yes

Defines ownership of View/Form, List, Query and Script definitions. Only new records are added to the target IJC_USER table. Existing records are never deleted nor updated.

IJC_CHANGE_LOG

No

We don't think it makes any sense to synchronize logging tables.

IJC_INVOCATION_LOG

No

We don't think it makes any sense to synchronize logging tables.

IJC_ITEM_INFO

No

This table is not used by current version of IJC.

IJC_ITEM_USER

No

This is IJC internal table used for locking. It must not be synchronized.

IJC_SECURITY_INFO

No

Contains security definitions (eg. LDAP or AD settings). This table could be synchronized, but currently it is not. The security definitions are fully under your control and you may want to have different security set up in both test and production environments.