SCMVolunteers

From Genunix

This space is to coordinate the volunteers helping out with the SCM Migration Training.

The Sun members of the SCM Migration team will be giving internal talks (Kernel Technical Discussions) to help with the transition from TeamWare to Mercurial.


Contents

[edit]

Mercurial Training

The Mercurial program is named hg (the chemical symbol for Mercury). Every Mercurial command starts with hg, followed by the command name, followed by any relevant options and arguments.

Just running hg on the command line displays a list of commands. 

$ hg
Mercurial Distributed SCM

basic commands (use "hg help" for the full list or option "-v" for details):

add        add the specified files on the next commit
annotate   show changeset information per file line
clone      make a copy of an existing repository
commit     commit the specified files or all outstanding changes
...

The help command can be used to get more information about any command. 

$ hg help diff
hg diff [-a] [-I] [-X] [-r REV1 [-r REV2]] [FILE]...

diff repository (or selected files)
...

Unlike TeamWare, Mercurial does not support user id based access control, though it does have an ACL extension. Repositories on opensolaris.org control commit access via ssh.

[edit]

Installation

You will need Python (the SUNWPython package) and Mercurial (SUNWmercurial) installed on your machine. You'll want Mercurial 1.0 or later (snv_88 or later).

The Mercurial Wiki has details on building Mercurial from source. Mercurial is primarily written in Python, but some of the extensions are in C.

[edit]

Getting Started

Unlike TeamWare, Mercurial supports cloning and syncing repositories over HTTP and SSH. 

hg clone ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate
hg clone http://hg.genunix.org/on.hg

Mercurial documentation generally uses the term repository rather than workspace; both terms refer to the same underlying concept. Like a TeamWare workspace, each Mercurial repository is self-contained. When you clone a repository, the new repository becomes an exact copy of the existing one at the time of the clone.

Note: Mercurial does not support partial clones at this time, but it is a commonly-requested feature; see the PartialClone entry in the Mercurial wiki for more information.

Quick Reference
TeamWare/SCCS/wx Command Mercurial/Cadmium Command Step-By-Step
bringover -p /ws/onnv-clone -w ~/ws/onnv hg clone /ws/onnv-clone ~/ws/onnv Setting Up A Child Repository
sccs edit file.c N/A You don't have to go through this step when using Mercurial, just start editing the file
sccs prs -e file.c hg log file.c Show all changes for file.c
sccs get -pm file.c hg annotate -n file.c Print file.c with the revision of each line
sccs info hg status .
sccs diffs file.c hg diff file.c Mercurial shows diffs in unified diff format
sccs delget file.c hg commit file.c or hg commit hg commit will commit all changed files, or just the files specified on the command line
sccs unedit file.c hg revert file.c
bringover -n hg incoming
bringover hg pull -u Syncing up with the parent
resolve -w ~/ws/onnv hg merge Merging conflicting changes
workspace parent hg paths Displays the parent repository
wx out N/A use hg status -mard to get a list of modified, added, etc files
wx list hg list (Cadmium)
wx redelget hg recommit (Cadmium)
wx reset file.c (not available) see bug 259
workspace parent -p /ws/onnv-gate n/a If you want to putback to a different parent you specify it when you push the changes, e.g. hg push /ws/onnv-gate
wx pbchk hg pbchk (Cadmium)
wx bugs hg bugs (Cadmium)
wx comments hg comments (Cadmium)
putback -n . hg outgoing -v hg outgoing lists outgoing changes, with the -v flag, it will also show affected files per-changeset
putback < comments.txt hg commit -l comments.txt; hg push It is the hg push that pushes the changes to the parent
wx backup hg backup (Cadmium)
wx restore hg restore (Cadmium)


[edit]

Setting Up A Child Repository

The equivalent of the TeamWare bringover command is the clone command which make a clone of a given repository.

1. To clone the onnv-gate parent repository, type the following command where my-copy is the name of your workspace.

hg clone ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate my-copy
requesting all changes
adding changesets
adding manifests
adding file changes
added 3196 changesets with 65633 changes to 42834 files
39543 files updated, 0 files merged, 0 files removed, 0 files unresolved

A child repository is created in the my-copy directory.

Note that this repository only includes the open source tree (usr/src). Internal developers can get the closed tree with

hg clone /net/onnv.sfbay.sun.com/export/gate-hg/usr/closed my-copy/usr/closed

(This command is likely to change once the gate has transitioned to running Mercurial.)

The nightly build script knows how to pull from and build both repositories. See the nightly man page (CLOSED_CLONE_WS environment variable) for details.

2. To edit a file, open it in you favorite editor.

vi file.c

Mercurial tracks the changed file.


[edit]

Editing files

There is no equivalent of the sccs edit or wx edit. Mercurial automatically keeps track of modified files.

[edit]

Adding files

The equivalent of the wx create command to add files to the workspace is the add command. 

hg add <file1.c> ...

These files will be added to the repository on the next commit.


[edit]

Examining differences

The equivalent of the sccs diffs and the wx diffs commands is the diff command. 

hg diff

Without any arguments, hg diff will print out the differences between all modified files in the repository.

Differences between files are shown using the unified diff format.

To see the differences between two changesets, use 

hg diff -r rev1:rev2

If any files have been renamed, you'll get more sensible results with 

hg diff -g -r rev1:rev2
[edit]

Committing your changes

The equivalent of the sccs delget and wx delget commands is the commit command. 

hg commit

Unless you use the -m or -l options, this command will invoke a text editor (configured using the EDITOR and HGEDITOR environment variables) where we can add a commit message (putback comment). Once we save the changes and quit the editor, the changes are committed to the repository.


[edit]

Syncing up with the parent

The pull and update commands are used to sync up the workspace with the parent. 

hg pull
hg update

The pull command pulls changes from the parent repository but does not actually make any changes to the files in the repository. The hg update command is used to actually update the files in the repository.

Note: These two commands can be combined into a single hg pull --update or hg pull -u command.

A repository location could be passed to the pull command to sync the repository with a different parent. 

hg pull --update /net/myserver/myproject-gate
hg pull --update ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate
hg pull --update http://hg.genunix.org/on.hg

Remember that the closed tree is in a separate repository. You can use nightly to avoid having to do multiple pulls by hand, though this is only going to work well for build-only workspaces (no local changes).

The equivalent of the bringover -n command (used to check what changes would be carried out without actually modifying the workspace) is the incoming command. 

hg incoming
hg incoming ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate
[edit]

Merging conflicting changes

Mercurial changesets cover the entire repository, rather than recording changes on a per-file basis. This means that when you pull from the ON gate, the odds are high that you will have to merge your changes with the changes in the gate, as in this example: 

% hg pull -u
pulling from ssh://anon@hg.opensolaris.org//hg/onnv/onnv-gate
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 3 changes to 3 files (+1 heads)
not updating, since new heads added
(run 'hg heads' to see heads, 'hg merge' to merge)

% hg up
abort: update spans branches, use 'hg merge' or 'hg update -C' to lose changes

At this point, you should run hg merge to merge the two branches. If you have uncommitted changes, you will need to commit them before merging (just like Teamware).

If your changes and the changes from the parent workspace do not conflict, the merge will happen non-interactively. If there are conflicts, Mercurial will invoke a merge application to ease merging the two changes.

Note that Mercurial's definition of "conflicting change" is different from Teamware's. Teamware considers two sets of changes in the same file to conflict, no matter where the changes are. Mercurial considers them to conflict only if they overlap. If your changes are at one end of the file and the parent gate's changes are at the other end, Mercurial will do an automated merge. So be sure to review all merged files carefully, to avoid mismerges.

Note for internal developers: if you used hgsetup to create your .hgrc, Mercurial will use FileMerge (if it can find it) to merge conflicting changes. This usage is different from Teamware in two ways. First, the parent and child panes are reversed compared to Teamware. Second, a separate FileMerge process is created for each file. So you'll need to exit FileMerge to get to the next file, rather than just clicking on Save.

In any event, once you have finished the merge, Mercurial will then prompt you to commit the merge. 

% hg merge
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
% hg commit -m "merge from onnv-gate"

It is rarely desirable to leave your workspace with more than one head (i.e., branched), so you should merge immediately after updating.

Mercurial will not allow you to push your changes, if those changes create a second head, unless forced. ON will not accept changes that branch the gate (think of this as a merge turd, only a good 700% worse).

[edit]

Putting back the changes to the parent repository

All changes up to this point have been in the working clone repository. The changes can be put back to the parent repository using the push command. 

hg push path_to_parent

Eventually this will look like 

hg push ssh://<opensolaris_userid>@hg.opensolaris.org/hg/onnv/onnv-gate

Where <opensolaris_userid> is your OpenSolaris.org login (you must have push privileges to the repository you are trying to push to)

This command is the equivalent of the "workspace parent -p /ws/onnv-gate" and "putback" TeamWare commands.

It is not mandatory to provide the path to the repository to push the changes to. If no repository is specified, the changes will be pushed to the repository specified by the 'default path' (defaults to the repository you cloned from, unless you use 'hg reparent')

The equivalent of the "putback -n" command is Mercurial's "outgoing" command. 

hg outgoing
hg outgoing ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate

If you wish to see the list of outgoing files, you can use Mercurial's template facility, e.g.: 

 hg outgoing --template "{rev}:{node}\n{author}\n{date}\n{desc}\n{files}\n"

Internal developers should keep in mind that the closed tree is in a separate repository. So if you have changed anything in the closed tree, you will need to do a second push.

[edit]

Downloading and Building the new onbld Package

If you don't want to build, you can install a pre-installed SUNWonbld package from: http://dlc.sun.com/osol/scm/SUNWonbld/

The latest packages are always available as:

http://dlc.sun.com/osol/scm/SUNWonbld/SUNWonbld-latest.i386.tar.bz2

http://dlc.sun.com/osol/scm/SUNWonbld/SUNWonbld-latest.sparc.tar.bz2

If you wish to build it yourself, you can clone from the SCM Migration gate: 

% hg clone ssh://anon@hg.opensolaris.org/hg/scm-migration/onnv-scm
copy and edit the opensolaris.sh into onnv-scm directory.
% bldenv -t <envfile>
% cd onnv-scm/usr/src/tools
% make install && cd SUNWonbld && make install


[edit]

Installing the New onbld Package & Setting up your environment

Download and install the package as mentioned above with pkgadd. Once you have the package installed, you can run /opt/onbld/bin/hgsetup to setup your environment.

hgsetup will create your ~/.hgrc and populate with all the settings needed.

If you don't want to or can't install SUNWonbld, you can also load it via NFS. There are two current *test* locations you can use. Both these are reinstalled every night with new onnv-scm bits, and aren't canonical paths - so use at your own risk:

For x86: 

 export PATH="/net/tonic-gate.sfbay.sun.com/opt/onbld:$PATH"

For SPARC: 

 export PATH="/net/stomper.sfbay.sun.com/opt/onbld:$PATH"

And point your ~/.hgrc files to load Cadmium as so: 

 [extensions]
 hgext.cdm=/net/server/opt/onbld/lib/python/onbld/hgext/cdm.py
[edit]

Trying It Out

The Mercurial ONNV repository mirror is located here: 

ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate

This is a clone repository. Only pull operations are allowed. It is a live mirror of the internal TeamWare onnv-gate workspace and is updated as putbacks happen in real-time. Subscribe to the onnv-notify mailing list to receive putback notifications as they happen.

The SWAN-internal ONNV repository (also read-only) is located here: 

/net/onnv.sfbay/export/gate-hg

Do *NOT* push to this repository.

[edit]

Moving an Existing TeamWare Gate to Mercurial

There is not yet an easy way to migrate a TeamWare gate to Mercurial. The onnv team uses tools under /ws/onnv-gate/public/bridge to track changes to the gate into mercurial in near-real time, but in general projects migrating are better off moving exactly once.

The SCM Migration Project has a script, wx2hg which will permit a TeamWare workspace (usually, but not necessarily a child of the TeamWare onnv-clone) to be converted to a Mercurial workspace.

You can also manually perform the steps that wx2hg does. First, get your TeamWare workspace and your virgin clone Mercurial repository to the same "head" putback/changeset. Then use wx backup on the TeamWare workspace. If you haven't done any renames, you can just restore the ?.clear.tar file into the Mercurial repository manually (don't use wx restore). If you have new files, be sure to register them with "hg add". If you have renamed files, things get a little trickier, and you're probably better off using wx2hg.

[edit]

Setting up a new (Mercurial) Project repository on OpenSolaris.org

This worked example is for a project targeting ONNV but a very similar procedure would be used for other upstream consolidations (such as SFW)

First create a local clone of the Mercurial onnv-gate like this: 

   $ hg clone ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate myproject

If you need to add already existing local changes see the *#Moving an Existing TeamWare Gate to Mercurial section.

You now need to create a repository on [opensolaris.org] to host this. In your project page there is an "SCM Management" link that is shown only to project leads. Click that. On the left hand nav-column there will then be a link "Add Repository". Fill in the form.

The Anonymous here means allow anyone to pull from the repository, if you don't tick that then only people with an opensolaris.org account with loaded ssh keys can do a pull (I generally allow it as do most projects I believe). Project leads can always do a push, and you can delegate that to people who are listed as observers too.

The name you give is tagged on the end of your project URL. So if you say "gate" you will end up with: 

   "ssh://hg.opensolaris.org/hg/myproject/gate"

The notification email gets every push message, so choose wisely what you set this too. Some projects use a dedicated -notify@ alias others just use their -discuss@ alias.

You are now ready to push your changes so lets configure your local copy of your Mercurial repository with the paths. Add the following to the .hg/hgrc file in your myproject dir: 

   [paths]
   default=ssh://username@hg.opensolaris.org/hg/myproject/gate
   default-push=ssh://username@hg.opensolaris.org/hg/myproject/gate
   onnv-gate=ssh://anon@hg.opensolaris.org/hg/onnv/onnv-gate

Now lets do the push: 

   $ hg push

You now have a populated repository on opensolaris.org, an email will be sent to the alias you specified in the repository setup. In most cases only a single email will be sent as they are combined into one message for a push with a large number of changesets.

[edit]

Related Documentation

[edit]

SCM Migration Project

Creating migration tools and converting existing development tools to work with Mercurial and possibly Subversion; see http://opensolaris.org/os/project/scm-migration/


[edit]

Todo

I don't think the hg commands are organized very well into "objects" and "operations" on those objects It might be helpful here to divide up the basic hg commands into sections like: operations one a repository, operations on a file, operations on a change set, etc, etc. It's hard because a lot of them touch multiple kinds of objects at once, but making the effort might help a new user to find the command they are looking for. I start to do this once, but didn't finish.

Retrieved from "http://www.genunix.org/wiki/index.php/SCMVolunteers"