RADE

Principles

Nowadays, many enterprises are international enterprises. So, people working on various places in the world must be able to work together, and the developed code has to be accessible and synchronized at any moment, and for people working on the same project, but in different localizations in the world.

SCM is designed for managing in one or several databases the objects produced by developers. When all users are connected over the same L.A.N. (Local Area Network) one single SCM installation is enough to manage all objects: this installation defines one site whatever the number of databases is [1].

When development projects take place over different geographic places, it is needed to install SCM on all places, therefore setting up several SCM sites [2].

If those sites are connected over a W.A.N., SCM provides inter-site commands for replicating objects between them. This set of commands will allow SCM to be used by several teams in different countries, as if they all were in the same site. The process becomes identical between people working in two different sites, at different hours of the day, and people working in the same office. SCM objects are going to be shared by several SCM sites.

This is the aim of those new commands: everybody can work as if they were in the same office, therefore, being able to work on the same products, promoting and synchronizing their data at any moment.

[Top]

Global Architecture

Inter-site exchanges involve two types of sites:

• The remote site: considered as being the "parent site" 

• The local site: considered as being the "child site".

Parent and Child Sites (or remote and local sites)

All the workspaces belonging to both sites can be seen as pertaining to the same global workspace tree; data are transferred from one workspace of a site to another workspace of the other site, and in both ways.

The local integration transfer workspace o the local site can be seen as being a child workspace of the parent remote workspace and the usual attachment, promotion and synchronization operations that exist within a local workspace tree are replaced here by inter-site commands.

Considering the "Site 2" as being the transfer initiator, then

on site 1: remote site

• The REMOTE_MIRROR_WS:
  • Function: this workspace is an intermediary workspace, reserved exclusively for the inter-site transfer. It will be created by the adl_mk_is_transfer command. No SCM action must be executed in this intermediary workspace. This workspace has no image. This workspace should not be deleted.
  • Responsible: the SCM administrator of the remote site

   

• The REMOTE_PARENT_WS:
  • Function: this workspace is the workspace which collect the promotion sent by the local_transfer_ws. This workspace can be either the root workspace, or an integration workspace (int ws), or any project workspace (prj ws)
  • Responsible: a developer, or a project manager, or the release manager of the remote site

on site 2: local site

• The LOCAL_TRANSFER_WS:
  • Function: this workspace is the workspace which attaches, synchronizes, promotes to the parent workspace. This workspace can be a root workspace, as well as a development workspace. The transfer workspace can be a root workspace if there is only one person working on the project; the use then, will be simple. If the transfer workspace is a integration workspace, for two or more development workspace, a dummy root workspace will be created, with specific parameters. See the second example for more details.
  • Responsible: the release manager or a developer of the local site

[Top]

Hardware and Client/server inter-site architecture

The figure 4 shows the hardware architecture, associated to an inter-site transfer. 

Figure 4: Hosts involved in inter-site transfers

The inter-site commands are based on a tcp-ip client/server architecture. 

On the parent remote site, a SCM inter-site Transfer Manager has been set up on one host and is waiting on a known port for remote clients to connect.

• the Transfer Manager should be located on a SCM client host server. It is not recommended  to run the transfer manager on the SCM server itself, for performances reasons.
• For more information on a transfer manager creation, see the Transfer Manager documentation

On the local site, any user might be able to start a transfer by executing the inter-site transfer's commands.

[Top]

Software inter-site architecture: inter-site commands

All the inter-site command's names are composed by the adl suffix, and most of them contain the "is" abbreviation ("is" stands for "inter-site). The others contain the word "transfer".

These commands are used by the local site to manage the inter-site SCM exchanges.

A set of administrator's commands:

• adl_mk_is_transfer: creates a transfer
• adl_rm_transfer: removes a transfer
• adl_ren_transfer: renames a transfer
• adl_set_is_transfer: updates transfer parameters
• adl_link_is_transfer: for multi-tree workspaces

A set of user's commands:

• adl_ls_transfer: lists existing transfers
• adl_ls_fw_is: lists attachable components
• adl_attach_is: attaches components from remote site
• adl_detach_is: detaches a component
• adl_sync_is: synchronises with the remote site
• adl_promote_is: promotes to the remote site

The figure 5 shows commands launched in a global workspace tree

[Top]

Creating an inter-site transfer

The inter-site creation is managed by the adl_mk_is_transfer command. Before creating the transfer, we strongly recommend you to read the following advices.

A transfer is created by the local site administrator, using the adl_mk_is_transfer command. For more details, see the documentation reference of the adl_mk_is_transfer command. The transfer's creation has to be prepared, on the parent remote site as well as on the child local site.

On the parent remote site

On the parent site, called also remote site, a workspace will receive the modification sent from the child workspace. This workspace can be either the root workspace, as well as a project workspace.

The SCM administrator on the parent site has to: 

• Start on the machine <host_name> the SCM inter-site Transfer Manager [3] by specifying the communication port <port> (adl_transfer_mngr)
• Give to the local SCM administrator the data he needs: transfer manager <host_name:port>
• Realize the necessary firewall update with the network team: allow the machine <host_name> to receive data on the port <port>

The user responsible of the parent's workspace has to: 

• Collect the modifications sent to the parent's workspace (adl_collect)
• Publish the modifications (adl_publish)

Here are three samples of a remote site workspace's structure: 

• In the first example, the local site is considered as a development workspace, exactly like the remote development workspace. They both promote in the remote project workspace.

Figure 6-1: architecture first example 

• In the second example, the local site promotes directly in the remote root workspace. In this case, the local transfer workspace is an integration workspace. All necessary controls must have been done on the local site, to verify there are no error.

  Figure 6-2: architecture second example 

  

   
• In the third example, the local site promotes to a remote project workspace. In this case, there are three developers, working together: two developers on the local site and one developer on the remote site. They are in the same team, and promote in the same remote project workspace. A dummy local root workspace must then be created. All necessary controls must have been done on the local site, to verify there are no error.

  Figure 6-3: architecture third example 

On the child local site

Before launching adl_mk_is_transfer, on the child site, called also local site, the local administrator has to: 

• Ask for a transfer manager <host_name:port> to the remote site administrator
• Ask for a remote parent workspace's name to the remote SCM site administrator and/or the remote release manager
• Realize the necessary firewall update with the network team
• Create the transfer tree and the transfer workspace (adl_mk_tree, adl_mk_ws, or adl_mk_root_ws)
• Choose a dedicated local host where the transfer will be created (via adl_mk_is_transfer), and the inter-site commands will be launched
• Create the transfer dedicated to the inter-site transfer workspace (adl_mk_is_transfer) on the dedicated local host
• Manage the inter-site transfer, using the following commands: adl_link_is_transfer, adl_ls_transfer, adl_set_is_transfer, adl_ren_transfer, adl_rm_transfer.

[Top]

Warning: the adl_mk_is_transfer command is a mono-tree command. This means for a multi-tree workspace, you have to:

• use the adl_mk_is_transfer command for one tree
• use the adl_link_is_transfer command for the others trees of the multi-tree workspace

Once this is done, all the inter-site commands are multi-tree commands.

Here are two complete transfer examples. One with a root workspace on the local site, and one with a development workspace on the local site; both transfers are done with a project workspace on the remote site.

Example with a root workspace on the local site

-1-  In the remote site:

• Creation of the tree MainRemoteTree

                        adl_mk_tree MainRemoteTree -db DatabaseName -cs ContentServerId ...

• Creation of the root workspace and the project workspace where the transfers will be done, with the shared frameworks

                        adl_mk_root_ws  RemoteRootWs ...
                        adl_mk_ws RemoteProjectWs -parent_ws RemoteRootWs  ...
                        adl_ch_ws RemoteProjectWs
                        adl_attach Fw1 Fw2 ... -mod

-2- In the remote site:

• Update of the firewall, in order to allow the transfer between the remote site and the local site. Responsible: the network administrator of the remote site.
• Start the transfer manager, on a dedicated server, named "ServerName", with the port number "PortNumber". Responsible: the SCM administrator.

To create the transfer manager, the following command has to be launched:

                        adl_transfer_mngr -port PortNumber -cmdfile ParametersFile

For more details and content of the parameter file, see the [5] "Getting started" article

-3- In the local site:

• Creation of the tree: LocalTree

                        adl_mk_tree LocalTree -db DatabaseName -cs ContentServerId ...

• Creation of the transfer root workspace

                        adl_mk_root_ws LocalRootWs

• Creation of the transfer

                        adl_ch_ws LocalRootWs
                        adl_mk_is_transfer -ws LocalRootWs -r_is_server HostName:PortNumber -r_parent_ws RemoteProjectWs

The transfer's name is not mandatory. A default transfer's name will be given if no transfer name has been chosen.

Example with a multitree development workspace on the local site

-1-  In the remote site:

• Creation of the two trees: RemoteTree1 and RemoteTree2

                        adl_mk_tree RemoteTree1 -db DatabaseName -cs ContentServerId ...
                        adl_mk_tree RemoteTree2 -db DatabaseName ...

• For each tree, creation of the root workspace and creation of the project workspace where the transfers will be done, with the shared frameworks

                        adl_mk_root_ws  RemoteRootWs1 ...
                        adl_mk_ws RemoteProjectWs1 -parent_ws RemoteRootWs1  ...
                        adl_ch_ws RemoteProjectWs1
                        adl_attach Fw1-1 Fw1-2 ... -mod

                        adl_mk_root_ws  RemoteRootWs 2...
                        adl_mk_ws RemoteProjectWs2 -parent_ws RemoteRootWs 2 ...
                        adl_ch_ws RemoteProjectWs1
                        adl_link_ws -parent_ws RemoteRootWs2 -tree RemoteTree2

-2- In the remote site:

• Update of the firewall, in order to allow the transfer between the remote site and the local site. Responsible: the network administrator of the remote site.
• Starting the transfer manager, on a dedicated server, named "ServerName", with the port number "PortNumber". Responsible: the SCM administrator.

To start the transfer manager, the following command has to be launched:

                        adl_transfer_mngr -port PortNumber -cmdfile ParametersFile

For more details, see the [5] "Getting started" article

See figure 5 for this example

-3- In the local site:

• Creation of the two trees: LocalTree1 and LocalTree2

         ---> local site administrator:      adl_mk_tree LocalTree1 -db DatabaseName -cs ContentServerId ...
                                                       adl_mk_tree LocalTree2 -db DatabaseName -cs ContentServerId ...

• Creation of the two dummy root workspaces with no image

         ---> local site administrator:      adl_mk_root_ws LocalDummyRootWs1 -no_image
                                                       adl_mk_root_ws LocalDummyRootWs2 -no_image

• Creation of the integration project workspace

  ---> local project administrator: adl_mk_ws LocalProjectWs -parent_ws LocalDummyRootWs1 -dir ...
                                               adl_link_ws LocalProjectWs -tree LocalTree2 -parent_ws LocalDummyRootWs2

   
• Creation of the two multitree development workspaces, one for each developer

---> developer number One:      adl_mk_ws LocalWsOne -tree LocalTree1 -parent_ws LocalProjetcWs -dir ...
                                             adl_link_ws LocalWsOne -parent_ws LocalDummyRootWs2 -all_trees
---> developer number Two:      adl_mk_ws LocalWsTwo -tree LocalTree1 -parent_ws LocalDummyRootWs1 -dir ...
                                             adl_link_ws LocalWsTwo -parent_ws LocalDummyRootWs2 -all_trees

• Forbid the local promotion for the project workspace LocalProjectWs, in order to avoid errors, and to realize promotions only via intersite transfers

---> local site administrator:     adl_set_ws LocalProjectWs -no_promote_cmd

• Creation of the transfer, from the local integration project workspace

         ---> local project responsible:    adl_ch_ws LocalProjectWs
                                                       adl_mk_is_transfer -ws LocalProjectWs -r_is_server HostName:PortNumber -r_parent_ws
                                                                                  RemoteProjectWs1 -tree LocalTree1 -no_cr
                                                       adl_link_is_transfer -ws LocalProjectWs -r_parent_ws RemoteProjectWs1 -tree LocalTree2
        

The transfer's name is not mandatory. A default transfer's name will be given if no transfer name has been chosen.

Using an inter-site transfer

The user responsible of the child's workspace has to:

• Go under the transfer workspace (adl_ch_ws)
• See the list of attachable  components (adl_ls_fw_is -all)
• Attach the components to be shared (adl_attach_is)
• Synchronize the workspace (adl_sync_is)
• Work in the workspace, or in the children workspaces, as used to (adl_co, adl_mk_elem, adl_mv etc...)
• Promote all the modifications (adl_promote_is)
• If needed, launch the adl_solve_merge_is command. For this particular case, see the paragraph below

Here are two complete transfer examples. One with a root workspace on the local site, and one with a development workspace on the local site; both transfers are done with a project workspace on the remote site.

Example with a root workspace on the local site

-1- In the local site:

• List of the available frameworks

                        adl_ch_ws LocalRootWs
                        adl_ls_fw_is -all

• Attach the desired frameworks

                        adl_attach_is Fw1 Fw2 ... -mod

• Synchronize and promote the modifications

                        adl_sync_is
                        adl_co ...
                        adl_promote_is

The user responsible of the parent's workspace has to:

• Go under the transfer workspace (adl_ch_ws)
• Collect modifications (adl_collect)

-2- In the remote site:

• Collect all the modifications

                        adl_ch_ws RemoteProjectWs
                        adl_collect

Example with a multitree development workspace on the local site

-1- In the local site:

• List of the available frameworks

   ---> project responsible:     adl_ch_ws LocalWs1
                                          adl_ls_fw_is -all
   

• Attach the desired frameworks, from concerned trees

    ---> project responsible      adl_attach_is Fw1-1 Fw1-2 ... -mod -tree LocalTree1
                                           adl_publish
                                            

• List of the available frameworks and attach them

         ---> developer number One:      adl_ch_ws LocalWs1
                                                      adl_ls_fw -all
                                                      adl_attach Fw1-1 Fw1-2 ... -mod -tree LocalTree1
         ---> developer number Two:      adl_ch_ws LocalWs2
                                                      adl_ls_fw -all
                                                      adl_attach Fw1-1 Fw1-2 ... -mod -tree LocalTree2

• Synchronize and promote the modifications

  ---> both developers              adl_sync           synchronization
                                             adl_co ...          modification
                                             adl_promote      promotion

• Synchronize and promote the modifications

  ---> project responsible           adl_sync_is        synchronize
                                             adl_collect          collect modifications
                                             adl_promote_is    promotion is done on both trees at the same time.

The user responsible of the parent's workspace has to:

• Go under the transfer workspace (adl_ch_ws)
• Collect modifications (adl_collect)

-2- In the remote site:

• Collect all the modifications

                        adl_ch_ws RemoteProjectWs
                        adl_collect

[Top]

Short commands reference

Here is an overview of the main commands. For more details, see each command reference.

 adl_mk_is_transfer

adl_mk_is_transfer is the command that allows the local site administrator to define a transfer with a remote site. This command is an administrator command.

For an inter-site transfer, necessary data are:

• the child site name
• the child site workspace name
• the child site workspace tree name
• the parent workspace name
• the parent workspace tree name
• the transfer name
• the server where the transfer manager runs

To launch the command: the administrator will be in the local transfer workspace.

Actions realized by the adl_mk_is_transfer command:

• If the local workspace is a multitree workspace, the command displays a message saying you have to use the "-tree" option if it is not already done
• If no transfer name is given, a default name is chosen: Transfer1, Transfer2, ... The transfer's name is unique for a given workspace
• The command verifies the connection (ping)
• The command verifies the existence of the remote tree
• The command verifies the existence of the remote workspace
• The command looks for the SCM remote workspace name ID
• The command verifies the existence of the local tree
• When the command is launched (one time only), it creates the report dedicated workspace (which has no image) on the remote site. It keeps in the local database all the parameters described just before.

adl_ls_fw_is

adl_ls_fw_is is the command that visualizes all the available frameworks for the transfer in the remote parent workspace. Those frameworks are shared between two distant sites, exactly as if they were shared by two persons working in the same team.

adl_ls_transfer

adl_ls_transfer is the command that visualizes all the transfers available on a given site. This command is a user command.

adl_rm_is_transfer

adl_rm_is_transfer is the command that suppresses the transfer given in parameter, on the current site. This command is an administrator command.

Actions realized by the adl_rm_is_transfer command:

• Removes all the data and transfer's tids relatives to the given transfer of the local site

adl_set_is_transfer

adl_set_is_transfer is the command that allows the administrator to update parameters of a given transfer, and if necessary, for a given tree. This command is an administrator command.

The parameters that can be updated are:

• the server reference: host name and port number (option:  "-r_is_server)
• the remote parent workspace name (option: -remote_parent_ws_name)
• the remote tree name (option: -remote_tree_name)
• the local tree (option: -l_tree)

adl_ren_transfer

adl_ren_transfer is the command that allows the administrator to rename the transfer's name. This command is an administrator command.

adl_link_is_transfer

adl_link_is_transfer is the command that links a transfer between several trees.

adl_attach_is

adl_attach_is is the command that attaches one or several of the available frameworks, so the developers can work on those SCM objects. This command is an user command.

Like for all SCM operations the replication of objects between sites is performed between workspaces and the granularity of exchange is the framework.

adl_detach_is

adl_detach_is is the command that detaches one or several of the available frameworks. This command is an user command, and cannot be launched in the root workspace..

adl_sync_is

adl_sync_is is the command that allows the user to synchronize the local transfer workspace with his remote parent workspace. All the modifications done in the remote parent workspace will be reported in the local transfer workspace.

Actions realized by the adl_sync_is command:

• The command verifies that the current workspace is one of the workspaces defined in the administrator transfer's table
• The command verifies that no promotion is running at the same time in the local workspace
• The command verifies if the local workspace is not already synchronized
• The command synchronizes the local workspace, and reports all the modification done in the parent remote workspace, in the local workspace
• The command publishes the updates

adl_promote_is

adl_promote_is is the command that allows the user to promote all the local modifications done in the local workspace, to his remote parent workspace.

Actions realized by the adl_promote_is command:

• The command verifies that the current workspace is one of the workspaces defined in the administrator transfer's table
• The command publishes the updates
• The command promotes all the frameworks

adl_solve_merge_is

adl_solve_merge_is is the command that resolves the merges if necessary, between the objects, if they have been modified concurrently in the remote workspace, and in the local workspace. This command must be launched in the local workspace.

This command will be necessary if there is a conflict between the remote mirror workspace and the local root workspace. Here are two cases that can happen:

- Case 1 -

• adl_promote_is  in the local workspace
• adl_rm req  in the remote workspace
• adl_sync_is in the local workspace
• another adl_sync_is later in the local workspace. In this case, if the synchronization is OK, than there is no problem. The synchronization can fail, if there is a merge to resolve in the mirror workspace and in the local workspace. To resolve this case, you have to launch in the local workspace an adl_solve_merge_is command

The adl_solve_merge_is command downloads the objects in the local workspace, resolves the merge, then sends immediately the result in the mirror workspace.

- Case 2 -

• adl_promote_is  in the local workspace
• adl_publish  in the remote workspace at the same time. In this case, the promotion is stopped in the remote workspace. Merge can happen, and they are not resolved then.

The adl_solve_merge_is has to be launched in the local workspace.

Handling Concurrent Changes

Like across workspaces belonging to the same site, concurrent changes may happen when sharing components between sites. 

There are two kind of concurrent changes: 

• those that are handled automatically by SCM: for instance when a developer changes the contents of a .cpp file while another one moves the file into another location; the result of these two actions will be the new contents at the new location.
• those that are conflicting changes because they are the same operation performed concurrently in two different workspaces:
  • two changes of one file's contents
  • two move or renaming of the same file or directory
  • two creation of two files having the same name and in the same location

The first kind of concurrent changes is automatically managed by SCM commands and there is no particular action to be done when sharing components.

The second kind of concurrent changes lead to solving merges [6] when the conflicting changes come in the same workspace. When sharing components, the solving of merges occurs always in the child site transfer workspace. When they occur, the user has to solve them all before being able to continue the transfer.

In fact the way data are exchanged between sites follow the same (default) rules as between parent and child workspaces on one site:

1. the child workspace synchronizes itself against the contents of the parent workspace
2. the synchronization may lead to some merges that must be solved in the child workspace
3. once all merges are solved, the child workspace is allowed to promote its changes to the parent workspace

[Top]

Advices - Recommendations

Workspaces

Here are some rules and advices to follow to take full benefits of the transfer capabilities:

• It is better to dedicate a workspace for transfers than using one devoted to other tasks (development, integration, etc) because it provides an isolated place where changes are better handled and it let time to users to check that changes can be forwarded and eventually to ask remote teams to fix some bugs and to start new transfers.
• On the local site, the local transfer workspace can be a root workspace. It can also be a project workspace, but the project developer responsible will have to follow the second example rules.
• On the remote site, either a project workspace or a root workspace can be the remote parent workspace
• A transfer is restricted to a tree on both sites, this means that if transfers are to be set up between multi-tree workspaces, one transfer will have to be set up for each tree (see adl_mk_is_transfer and adl_link_is_transfer).
• All objects contained in the remote parent workspace can be exchanged. Then 
  • the first transfer may have to transfer a large amount of data if no one is already present in the target site
  • further transfers will handle the delta of changes, that is to say that each time a transfer is started, it compares the whole current workspace's contents against what was in it for the previous transfer, differences found are to be transferred.

Frameworks

Here are the steps to go through before being able to share components between sites.

• Think about the set of frameworks that needs to be shared. This set may lead to involve several workspace trees and several workspaces, so different transfers may be needed to share those components.
• Think about the structure of components on both sites: is there a particular installation on one site regarding the setting of a customer checker library [4]? In that case we must warn that no checker is called when exporting/importing objects between sites and then that both sites must share the same structure (even if SCM commands can handle different structures, it may not be the same for other tools).
• Think about the way components will be shared and integrated in both sites. 
  • is there a site which will be the supplier of the other? The supplier will be the child site and corresponding teams will use the inter-site transfer's commands to exchange data.
  • does a team will have to take the lead of a development? In that case, the corresponding site will be the parent site and will be the site where the SCM inter-site Transfer Manager will be started.
• Decide under whom identity the SCM inter-site Transfer Manager will be started. Remember that it will execute SCM commands on Parent site's workspaces. Decide on which host it will be started: this host must be reachable from child site's hosts and SCM processes running on it must be able to connect to the other SCM daemons (monitor, file server).

[Top]

In Short

Sharing components between sites consists in using the adl_is_xxx commands which establishes a connection with the SCM inter-site and shares data between different sites. Exchanges are done between usual SCM workspaces. The user who runs those commands is responsible for solving any conflicting change between the two sites.

[Top]

References

History

Copyright © 2001, Dassault Systèmes. All rights reserved.