Hit the 4gb database max size, what now?

[PolishRenegade]

Good day,

We are using the embeded db that comes with PlasticSCM and hit a slight problem...

Apparently we cannot go over the 4gb per repository limit so I'm wondering what are the options? Is it possible to partition a repository into multiple files?

Is our only solution to archive the repo and lose all logs/changesets? We have some pending merges that need to be addressed however the server wouldn't allow for them, can we free up space somehow?

As you can guess, I'm not much of a server administrator however I'm open to suggestions. I've searched a bit and migrating to a dedicated MySQL server wouldn't resolve this issue, unless I've read wrong.

Thank you for any and all help.

**Edit; I've been re-reading the administrator guide and there is a revision archiving functionality (I mistaken it with the repo-archiving one).

If I were to archive all revisions up to a certain point, could that be possible? Or do I need to archive specific revisions?

Pseudo Exemple; cm archive *#br:/main#revno<100 -c=”all files of all revs under 100” -f=”/archivedRevs/Project100”

- Will * as a wildcard work for all files?

- How to specify all revisions under, per say, 100?

I'm not sure how the commands work, couldn't find an exaustive list of them, only the quick start guide. From sniffing through the cm help I'm thinking it would look like this;

cm find "revs where revno < 100" --format="{item}#{branch}#{revno}" --no total | cm archive -c="all files of all revs under 100" -f="project100"

- In the 'cm help archive', the "comment" arg should be -c, however in the exemple it is -comment, which one to use?

- What is the purpose of the --format argument? Will omiting it affect the archiving or is this argument responsible of including "items/branches/revno" into the archive?

**Edit2; I've tried the following command line without success...

cm find "revs where changeset < 95 on repository 'default'" --nototal | cm archive -c="Core 104 Archive" -f="Core104" -

It returns;

"Incorrect object specification 4 26/09/2012 15:09:49 dir all /#br:main#0 Root dir

Not sure how to fix that...

** Edit3 and permission problem;

The command help is quite outdated but thanks to this thread; http://www.plasticsc...__fromsearch__1 I was able to run this command:

cm find "revision where changeset = 1 and type !='dir' on repository 'default'" --format="rev:{item}" --nototal | cm archive -c="Archive" -f="Archive_ChangeSet1_0" -

Newbie Tips for other people like me;

- You cannot archive directories thus the " type !='dir' "

- You cannot archive multiple revisions if files were moved in previous revisions

- You need to switch your workplace to the archive revision because moved files will throw an error saying the file doesn't exist

- The format is very important, I guess my command line inexperience assumed "cm archive" will take any revisions found by "cm find". Of course it's just a reference to the actual returned string which needs to be perfectly formatted, thus the batch... silly me.

Now I have all the achived files, over > 1000 of them actually... Problem now is that it tells me that I don't have the permissions to archive revisions. This makes no sense as both the server and repository is set as "All Users".

Also, it's quite a bunch of archived files to add into the externaldata.conf file, will have to find a way to automatically list them and add them into the conf file...

[PolishRenegade]

I think it would be usefull to also add the rational of archiving revisions instead of going through file size;

We are working with the game engine Unity and thus have a lot of binaries / text scenes. It is quite frequent for the developpers to switch back and forth between revisions of the projects as debugging and scene comparisons need to be done. Having working parts of the project under archive (scene, lightmaps, some textures, big models) and others not, would probably be problematic in the long run when compiling or merging those parts. (Merging an archived scene from revision X with a non archived scene from revision Y, is that even possible?)

Thus I think archiving the entire project up to a certain revision (which we know we won't revisit / change any time soon) would be safer.

Thank you,

[manu]

Hi PolishRenegade,

the default SqlServer CE backend is restricted by the 4GB databasefile "feature", so, what you need to do now is move your data into a new backend, which one? I do recommend you MySQL.

First you have to install a MySQL in the machine and the you need to migrate the data to it.

Here you have an article explaining how to migrate the information from a SqlServer CE to a new MySQL: http://codicesoftware.blogspot.com/2012/11/migrate-database-backend.html

The process is pretty simple and it only requires a MySQL installation.

[PolishRenegade]

Thank you for answering,

I followed the instructions in the link provided;

1) Stopped PlasticSCM Server

2) Migrated (MySQL successfully installed beforehand and test connection was sucessfull)

3) Started PlasticSCM Server

However all my data / changesets are lost. There was no error.

Due to hitting the 4gb limit and the archive command not working for me, I messed arround with the configuration files of db.conf.

This made the CE server not work, in this case, it would tell me the limit is 4091mb that my assigned value of 4092 is out of bounds. The reason was that cm archive was returning "The database file is larger than the configured maximum database size. This settings takes effect on the first concurrent database connection only. [ Required Max Database Size (in MB; 0 if unknown) = 4092 ]". This is why I set the db.conf to 4092...

However, the migration tool didn't catch that and just made a clean db without errors. I reverted back to the old CE db.conf, made sure everything was fine and re-migrated with a unique prefix.

It works, it's all good! I feel like a horrible newbie but apparently PlasticSCM is 90% newbie proof.

[manu]

Hi!

Can you please check if the MySQL databases are having the information?

You can check it with the MySQL manager or the MySQL command line utility

If you want to go back to sqlserver CE you just need to restore the db.conf file with the following content:

<DbConfig>
<ProviderName>sqlserverce</ProviderName>
<ConnectionString>DataSource={0};Persist Security Info=False;Max Database Size=4090;Max Buffer Size = 4096;Password=;</ConnectionString>
<DatabaseSuffix></DatabaseSuffix>
</DbConfig>

[PolishRenegade]

Manu,

Thanks for the follow up. As per my edit (didn't see your post before editing, sorry), everything works fine now!

[wvd_vegt]

Hi,

Just for the information, there are some tools for MSSQL Compact (like compact view and some commandline tools) that allow you to compress/clean the database and buy you some time (so you can plan a migration ;-). It sometimes takes off a lot of fat from a *.sdf file.

regards

[PolishRenegade]

The MySQL installation (with the installer) was quite plainless actually. I vaguely remember it being quite more complicated 7 years ago when I was doing some web dev.

The guides did help too.

Thanks for the advice, it's always good to know such things but I wouldn't had gone with the embedded db in the first place if I knew there was a 4gb limit. (started using PlasticSCM before the november article about migration)

Important thing is that I learned from this so it's a positive experience. (The artists didn't like not having access to PlasticSCM though...)