Using Xlinks with multiple Unity projects

[SynthoidCommander]

Our team has been using plastic for a while now, and as we begin to work on multiple projects, our common code shared between projects begins to fracture. To avoid this, we are experimenting with Xlinks. Setting up the actual Xlinks to be used in each project is simple enough, but the real trouble comes from the meta files generated by Unity once the linked files are brought in.

We don't want these meta files to transfer between projects, nor can we just tell Plastic to ignore meta files on each project. I've tried telling the repo for our shared code to ignore .meta files, but the files still seem to be committed and pulled in, even if the workspace for the shared repo is ignoring .meta files. Deleting the meta files in the main Xlink branch is a problem, since any sub-branches for individual projects would then have their meta files deleted when merging from the main branch again, breaking script references.

I've attached a simplified example of the layout generated for the Xlink repo as we link projects. You can see how edits made in one project should be merged to the main branch and then other projects, but that also includes meta files that cause issues in projects they did not originate in.

Basically, the problem is:

Meta files are needed for each project to function properly, but will break projects when shared, so we need a way to allow meta files in the individual projects themselves, but not the shared repo.

[SynthoidCommander]

Further elaborating on the topic, it seems the Xlink repo configuration files (ignore.conf, etc) are overlooked in favor of the specific project's configuration files. This explains why .meta files are committed from projects, and are pulled into the main Xlink branch when a file under the Xlink is changed, but don't show up in the Xlink repo's pending changes tab. (They are still in the "Items in workspace" tab though).

Forcing the meta files below the Xlinked folder to not commit to the Xlink repo, but to commit to the project's repo could solve the issue. Can such a thing be done?

[manu]

Hi!

just to make sure I understood the scenario.

You want to commit the meta files for the main project and not for the meta files under the xlinked repositories, right?

[SynthoidCommander]

We need the meta files to commit to the main project, but not commit to the Xlinked repositories.

The issue is we can't ignore the meta files completely since the projects need them to work, but committing meta files to the Xlink repo will cause cross contamination.

[SynthoidCommander]

Another possible scenario is the main branch in the Xlink repo doesn't have meta files, but the individual project branches do. The main goal is to isolate project meta files, either in the project repo, or on the project's branch in the Xlink repo.

[manu]

Understood,

let me try a few things and I'll come back.

[manu]

I think the easiest and best option is the following one.

Create a "hidden_changes.conf" file at the root directory of your main project. This file should contain something like this:

xlinked_Project1/**.meta
xlinked_Project2/**.meta

Additionally you'll want to have another file called "ignore.conf" with the same rules.

The first file, "hidden_changes.conf", will ignore all the changes done inside the projects directories (recursively) for the "meta" files. The pending changes view will not display those changes so you won't be able to commit them.

The second file, "ignore.conf", will avoid asking you if you want to add to the repository new meta files created under the project directories.

[SynthoidCommander]

These files go in the project root for the Xlink project? Or for each project linking to the Xlink repo?

[manu]

For each project linking external projects

[SynthoidCommander]

This gets into the frustrating dependency on meta files for Unity.

It looks like the conf files shown would ignore all meta files under the Xlink directory when committing, which is a problem.

If we don't commit the meta files at all, Unity will not properly associate scripts with game objects.

The issue then is that we have to commit meta files, but we don't want to commit meta files to the main branch of the Xlink repo. Every project relies on meta files to function, but meta files from other projects cause issues.

 

[manu]

Ok, you can then configure your xlinks and your main project like this: http://blog.plasticscm.com/2014/08/how-to-share-engine-repository-between.html

So you can commit what you want using child branches and not using the xlinked repositories main branches.

 

[SynthoidCommander]

That's actually how we are setting up the projects, the main issue is that when a project's Xlink branch is merged up into the main Xlink branch to allow other projects to access that update, it brings in meta files.

If we delete those meta files in the main branch so that other projects are not affected, merging from main back to the project branch we originally merged up from at some point in the future will cause problems (best case plastic evil twins, worst case the meta files are deleted in the target project, breaking the project).

In other words, we are using separate branches in the Xlink repo for each project, but we do not have a way to handle each project's meta files properly. (Each project's meta files are committed and will attempt to merge to main since they are already committed and cannot be ignored).

[Wolfram]

Long-time Unity user here, new to Plastic.

@SynthoidCommander: why do you belive the .meta files are incompatible between xlink clones? Let me elaborate on this:

The major reason why meta files are important (besides texture settings etc.) is that they contain the unique guid which allows referencing an asset anywhere even when it is moved or renamed, or temporarily not present (for example in package exports). As long as this guid is the same for each shared asset in any of your projects, there shouldn't be a problem, and it will even allow you to use assets from project B in project C, without losing the references in scenes and so on.

Prerequisite for that, though, is that your original repository (=your xlink source) already contains the original meta files, and is solely responsible for them. If that repo did not have (visible) metafiles in source control, for each xlink copy Unity will indeed create its own meta file, with guids that are most certainly different for each copy, and chaos will ensue.

Judging from your "the real trouble comes from the meta files generated by Unity once the linked files are brought in" in your first post, this is the problem you are having?

It is absolutely vital that any asset X in any project B, or C, or the original xlink source A has the same guid throughout. Having the meta files under source control in your xlink source should ensure that.

If your assets already have different guids, it will be difficult (but possible) to recover from that. The best approach is probably to use a search&replace tool that can replace the incorrect guids in projects B, C, ... with the original ones from xlink source A, for each file in the Assets subtree (including all meta files!). We've been using this fix in the past - it should be quite safe and works wonders.

[SynthoidCommander]

@Wolfram As you said, the main conflict was trying to backport the Xlinks to existing projects. I was hoping there would be a way to avoid meta files in the Xlink repo as it is mostly comprised of scripts, which for the most part only use meta files for keeping track of whatever references said script.

After messing with the various projects, I was able to get things under control, it just involved deleting a few .private.0 files generated from .meta files that did not exist in a project and were imported as regular files, instead of meta files. As well as some creative replacements to keep the existing script references intact.

Going forward, I don't think meta files will be as much of an issue, as they should properly share info. It is just tedious that backporting required so much brute force file management for the existing projects.

Overall, Xlinks seem to work fairly well once the initial set up is properly dealt with. Especially when starting new projects, which will require relatively little setup to get Xlinks working properly.

[Johan]

We had exactly this discussion a while back, how we should handle .meta files in our shared repositories. We finally agreed on simply letting the .meta files exist on the shared repository, so every project share the same meta data when it comes to the assets in the shared repo. As long as the initial contributor makes sure to commit .meta files along with the new files he creates in the shared repo, there aren't any issues.

I understand the issue when initially merging shared code into a shared repository, that's not pretty. But going forward, i.e. when starting a new project using the shared repo, we've found the simplest solution be to share the .meta along with the assets in the shared repo.