3.x to 4.1.x migration

[SilverKnight]

I ran across this, but wasn't sure it applied to my repository:

 

Restrictions due to branch inheritance

There are some important changes between 3.0 and 4.0 underlying structure which will force the 3.0 fast-export command to "discard" some branches.

What does it mean? Basically: branches inheriting from "last" won't be exported by the 3.0 fast-export command and hence won't be imported in 4.0. If this is a problem for you and prevents you to migrate to 4.0, please let us know.

 

What I'm trying to do is move my CVS repository with all branches into Plastic. I successfully used the guimport with the 3.0.187.38 server - all branches and code appear to be present. Tried to do a fast-export and ended up with only the main branch (6gb CVS repo created a 22GB Plastic 3.x DB and fast export produced a 234MB file)

 

Is it actually possible to move this into Plastic 4.x, or am I stuck with 3.x?

 

I also posted this as a quick/chat to plasticsupport, but don't know if that system is actually active (it appears to be a complete different site within codicesoftware), but thought I would post it here and see if the community had an answer (I expect that someone out there has already done something similar during their upgrade path)

[calbzam]

Hi, 

 

This is the easiest way to migrate a CVS repository to Plastic. We need git installed because we will use cvsimport command. I will try to explain you the steps with an example:

 

First of all, we have to be sure CVSROOT environment variable points to the CVS repository directory. A good choice is:

setenv CVSROOT /usr/local/src/cvsroot

 

You may need  to checkout a repository  from your  server or a collaborating web like Sourceforge :

e.g. cvs -d :pserver:username@projectname.domain.net:/cvs checkout <REPO>

Once we have the CVS repository in a local folder, we create a new folder to store git repository.

   e.g mkdir gcode-git

Now  “cd gcode-git” and we can execute “git cvsimport” command.  A “.git” folder is generated.

e.g git cvsimport  -v -d :local:/home/plastic/Downloads/gcode GC

 In the example command we indicate: path(/home/plastic/Downloads/gcode) and repository name (“GC”).    

If we execute “git branch”, we can check that all the branches are exported to git fine.

We have now our repository in git. At this point, we only have to create a git fast-export file (e.g repo.fe), and import the file to Plastic, indicating Plastic server and repository name we want to create.

 e.g  git fast-export --all -C repo.fe
 e.g  cm fast-import myrepo@localhost:8087 repo.fe

A post will be released very soon on our blog, with more info and more examples.

 

Best regards,

Carlos 

[SilverKnight]

@calbzam

 

Well, I suppose that might work - if this were a Linux based setup --- however, it isn't, it is an all Windows environment.  While it is great that plastic is moving along the path of "nice" integration with Git and other SCM projects, my question was moving from Plastic 3.x to 4.1.x - I have ALREADY imported my CVS repo to Plastic 3 (which took almost a week for a single project) - now I want to move it to Plastic 4.1.x.  I'm not really interested in re-exporting it to another tool (unless that is the ONLY option)  and as indicated, it should be FROM PLASTIC 3.x (NOT CVS) --> something else (git) --> Plastic 4.1.x if necessary.  (That's not to say I didn't try the path you outlined above - I just hit a brick wall on that one as outlined below)

 

If there is a git plasticimport for the 3.x version, I would give that another try - however, the plastic 3.x fast-export command does not export all branches, so I doubt there is a functional command here.

 

Thanks for the reply though.  Hopefully, other users may benefit from the information.

 

 

This is from the mysisgit release notes:

Git Release Notes (Git-1.8.1.2-preview20130201)

Last update: 1 February 2013

 

Introduction

 

These release notes describe issues specific to the Git for Windows release.

 

General release notes covering the history of the core git commands are included in the subdirectory doc/git/html of the installation directory. Look for files starting with RelNotes.

 

See http://git-scm.com/ for further details about Git including ports to other operating systems. Git for Windows is hosted at http://msysgit.github.com/.

 

Known issues

• Some commands are not yet supported on Windows and excluded from the installation; namely: git archimport, git cvsexportcommit, git cvsimport, git cvsserver, git instaweb, git shell.

 

 

It appears the other option "cygwin git" is a ridiculous package download and build-it-yourself affair - this site gives you a glimpse into the nightmare of installing the Cygwin Git "packages" on Windows.

 

http://blog.laranjee.com/how-to-install-git-on-windows/

[cidico]

Hi SilverKnight!

 

It's very weird that you cannot export all the other branches but main.

Can you see if you have the permissions in the other branches?

I remember that there was some special permission to do some actions in plastic.

Just right click on the branches and go to permissions and see what you can do there.

 

Also, why don't you try to fast-export your repo using the --no-data parameter and try to re-import it on 4.x?

The --no-data will make it much more faster and then you can see what's been exported from your Plastic 3 installation.

 

Hope it helps!

[SilverKnight]

I'm not sure there could be permissions on each individual branch different from the trunk on an import - I did check a few, and they are ALL set to allow (I cannot uncheck any of the allowed items, though I can check Denied boxes on different branches - I think I set the permissions at the repository level so that fast export would function - I had to check "advancedquery" so I believe I selected the check for "all allowed" and applied them to the repository).

 

I previously tried the --nodata  - here is an excerpt from output:  (there are a lot of these)

 

Discarding branch /main/build_4_8_0_11. It is not smart or it starts from LAST
Discarding branch /main/build_4_8_0_31. It is not smart or it starts from LAST
Discarding branch /main/build_4_8_0_21. It is not smart or it starts from LAST
Discarding branch /main/build_4_8_0_51. It is not smart or it starts from LAST
Discarding branch /main/build_4_8_0_41. It is not smart or it starts from LAST
Discarding branch /main/build_4_5_0_626. It is not smart or it starts from LAST
Discarding branch /main/build_4_8_0_61. It is not smart or it starts from LAST
Discarding branch /main/build_4_5_0_624. It is not smart or it starts from LAST
Discarding branch /main/build_4_5_0_625. It is not smart or it starts from LAST
Discarding branch /main/build_4_5_0_628. It is not smart or it starts from LAST
Discarding branch /main/build_4_5_0_629. It is not smart or it starts from LAST
Discarding branch /main/build_4_7_0_64. It is not smart or it starts from LAST
Discarding branch /main/build_4_7_0_54. It is not smart or it starts from LAST
Discarding branch /main/build_4_6_0_11. It is not smart or it starts from LAST

 

 

Our current build process creates a branch in CVS for each build that we do (in case we need to go back to a specific version to create a patch. That way, we don't introduce new features to fix a bug)

Given the branches show up in the 3.x repository, I can see that the guimport application transferred everything over from CVS.  I can also check out specific branches from this repository and have not had any issues with permissions getting them.

 

Branch exporer shows 456 items.

 

After seeing the above "discarded" branches, I started searching and found the document above indicating that "...force the 3.0 fast-export command to "discard" some branches." and noticed that it appears to be actually dropping those branches during the export.  I didn't see any way to force it not to drop them and was hoping there was some extra command/switch I could give it to grab all branches, even it it took longer due to structure, or took more space.  As indicated, my CVS respository only takes 6gb on disk where the 3.x database is ~3 times that.  Space really isn't an issue, even if it took 60gb to export it all to the 4.x format I would be ok with it.  Or, if it took another 5 days to export from 3.x (like it did to import it from CVS) -- the code base is actually about 575mb on disk, so I would an fast-export would be at least that size for a singe branch (however, as noted, it isn't... it is about half that - probably becuase it is only picking up the original HEAD branch from CVS and not the more recent/larger branch(es)).

 

I expect that if I can't get this to work, our company will end up going to TFS since we already have an MSDN license for that.    Yuck.  Half of our team has already migrated away from Jira to use the MS TFS test manager suite (which I think sucks in comparison to Jira) -- the other half never migrated away from Bugzilla.

 

I did a couple of informal tests for speed - CVS takes 15-20 minutes to check out a branch of this project.  Plastic does the same job in about 2 minutes.  That saves me about 13-18 minutes where I can actually do work instead of twiddle my thumbs.  I'd like to be able to present that to the teams here as an incentive to switch.  That time also adds up doing a build - it currently takes about 1.5-2.0 hours to check-out, build, check-in, and tag the project.  I would expect using plastic would chop about an hour off that (at least I hope so).

[calbzam]

Hi, 

I´m not sure the document you mentioned is this one: http://www.plasticscm.com/infocenter/technical-articles/kb-migrating-from-plastic-scm-3-to-plastic-scm-4.aspx . 

Restrictions due to branch inheritance:

There is a big difference between the changesets in 3.0 and 4.0. In 4.0, each changeset points to an entire, statically resolved tree. It wasn't true in 3.0 for the changesets on branches inheriting from "last" since "last" would dynamically move.
What does it mean? Basically: branches inheriting from "last" won't be exported by the 3.0 fast-export command and hence won't be imported in 4.0.

The thing is that after you migrated to Plastic3 from CVS,  all your branches are inheriting from "last" and they are not exported to Plastic4. There is a possibility to indicate manually a different father, modifying and checking branches one by one, but I suspect you have a lot of branches and it will be a tedious process.

 

In my opinion, the best option is to perform the migration from CVS to a system like Git or Mercurial and then executing fast-export, migrating all the branches to Plastic4.

This document explains how to perform a migration from CVS to Mercurial : http://mercurial.selenic.com/wiki/ConvertExtension

This post mentions that using cygwin git allows you to use git cvsimport command: http://stackoverflow.com/questions/2265221/can-we-use-git-cvs-on-windows

I´ve found also a resource to generate CVS fast-export files, but I haven´t tested it: http://www.catb.org/esr/cvs-fast-export/

 

Best regards,

Carlos

[SilverKnight]

I'm exploring this one - so far, I can run 4-5 of these per day - and they always crash at pass 16 building the mercurial repositiory with some error on binary files.

I've got about 4500+ change sets - the crashes started around 400 and the last run produced a crash processing changes set 800+ -- I've been deleting the files from a CVS copy of the files causing the crashes - However, as soon as it gets to a file that I need (and since every file that it has crashed on has been a binary file, I have little doubt that it will crash on some file that I actually need), I expect this option will also be a failure.

 

Even this was not a simple install... Had to install Mercurial, Python 2.7, then had to actually download a .ZIP containing the CVStoHG python script.  I notice that there is no FAST-EXPORT command either - that appears to be in another .zip file as a pythong script - who knows if that is even going to work.  Still waiting on getting a usable mercurial repository.

This document explains how to perform a migration from CVS to Mercurial : http://mercurial.selenic.com/wiki/ConvertExtension

 

This one is not going to happen.  I briefly looked into the cygwin stuff, and I already mentioned what I thought about the usability of the Cygwin suite.

This post mentions that using cygwin git allows you to use git cvsimport command: http://stackoverflow.com/questions/2265221/can-we-use-git-cvs-on-windows

 

This doesn't work on Windows - it is linux only - also no help for my shop.

I´ve found also a resource to generate CVS fast-export files, but I haven´t tested it: http://www.catb.org/esr/cvs-fast-export/

 

Best regards,

Carlos

 

As a side note, I considered CVStoSVN - but from everything google tells me, the cvstohg is the cvstosvn pythong script - which doens't seem to be working with my CVS repo anyway.  Yet another migration tool/route that isn't going to work.

 

So far, the ONLY tool that was able to successfully get my project out of CVS was the GUIMPORT.exe from Plastic 3.  Unfortunately, that didn't let me migrate it to 4.1

 

Maybe someone should build a Linux VM with all the necessary junk and provide a script that can be tweeked with a path to a copy of the CVS project/repository to be converted (or take a command line arg for that one).  That would make it easy. 

• Download the VM
• copy your CVS repo/project into this directory
• run this script with your project path (which does all of the conversion, including a fast export)
• then copy this fast-export file back to your machine after the conversion.

If there was something like that, then this conversion thing wouldn't be so bad.

So far, two weeks into conversion and still no dice.