non-recursive inclusion/exclusion with ignore.conf

[Wolfram]

Hiho.

Our global ignore.conf for Unity projects looks something like this (I removed several other entries not relevant to this post):

!ignore.conf
/Library
/ProjectSettings/ProjectVersion.txt
*~

• /Library contains workspace-/Unity-project-local files we don't want to be in version control.
• Same for /ProjectSettings/ProjectVersion.txt : We do need /ProjectSettings/*, but that particular file contains information about the local Unity project workspace.
• Many editors store backup files by appending a "~",  so we ignore all of these (these files are also ignored by the Unity editor).
• I'm actually not sure if we do need the !ignore.conf (or maybe better, !/ignore.conf), but it's there since we initially setup our server, and it's not causing problems.

Now, Unity is switching to a powerful package system called UPM (a variant of the npm Node package manager), so that the Package Manager in the Unity editor can handle installation and upgrade of software components and assets.

Unity stores installed packages in /Library/PackageCache, and the configuration in /Packages/manifest.json. Our ignore.conf is still good for that, because we only want the configuration, but not the actual package content (Unity takes care of this itself).

 

The problems come when developing your own packages. We store them in say /MyPackages, and we can tell the Package Manager to look for them there, as if they were regularly installed.

However, the packages can (and will) contain two special folders, "Documentation~" and "Samples~", with a "~" at the end, and these folders (and their content) need to be in version control, along with the rest of the package. The "~" cannot be omitted,  as this would make the folders visible to Unity, which will cause errors due to duplicate files/IDs. 

So my first approach was, "do not ignore these particular folders":

!ignore.conf
!Documentation~
!Samples~
/Library
/ProjectSettings/ProjectVersion.txt
*~

However, the result of this is that on the one hand, the other ignore.conf rules are NO LONGER APPLIED to any file having "Samples~" in its name. In particular, suddenly files like "/MyPackages/packagename/Samples~/old.tx~" show up in the pending changes, which I still want to exclude.

On the other hand, for installed packages these folders (and their content) suddenly show up as /Library/PackageCache/packagename/Documentation~/readme.md and so on, but I STILL want these to be excluded.

 

So how would I have to setup our ignore.conf to:

• ignore ALL files and directories ending in "~", EXCEPT directories named "Samples~" and "Documentation~" anywhere in the tree (or a possible simplification: ONLY in the /MyPackages/* tree)
• but at the same time STILL ignore EVERYTHING in /Library, even if there are directories named "Samples~" and "Documentation~"?

 

(I did check the evaluation order in https://plasticscm.com/book/index.html?utm_source=plasticscm-blog&utm_medium=blog-post&utm_content=configuringignoreditemsinyourworkspace#_pattern_files but cannot find a working solution).

 

Thanks!

[Héber]

Hi Wolfram,

That's quite a special setup you have there! I'll try to explain how to do it as best as I can, and provide you with the rules I think will let you have the ignore list as you wanted.

Your solution was not working because the recursivity you were using applied only to the path rules, leaving out name rules. This is a difference between our ignore.conf and gitignore.

First of all, you will need the /**~ rule. You need this instead of /*~ because this one ignores directories, but not the contents inside of it.

Then, you add the following ones:

• !^/[^Ll].*/Samples~$
• !^/[^Ll].*/Samples~/.*[^~]$
• !^/[^Ll].*/Documentation~$
• !^/[^Ll].*/Documentation~/.*[^~]$

These rules unmatch Samples~ and Documentation~ directories and its contents, unless the path starts with "L" or "l",  so you ignore the ones in Library.

After unmatching those rules, we want to ignore the whole Library folder, so we add a rule for it: /Library 

So, the whole file should be something like this:

/**~
!^/[^Ll].*/Samples~$
!^/[^Ll].*/Samples~/.*[^~]$
!^/[^Ll].*/Documentation~$
!^/[^Ll].*/Documentation~/.*[^~]$
/Library

Just for your information, note that this will work:

/src
!/src/foo
/src/foo/bar

But this will not:

src
!foo
bar

It is due to the name vs path recursivity I said above.

Hope this helps you, and feel free to ask anything you need.

 

Best regards,

Héber.

[Wolfram]

Ah, I wasn't aware that the exclusion-! can also be used for regex lines, nice!

Also it's great that you appear to be using a "complete" RegEx parser, instead of just mimicking the most common expressions. This way, I was able to use (?!Library/).* instead of [^Ll].* , using "negative lookahead", to explicitly only exclude "Library", as opposed to excluding any folder that starts with an "L".

A question: The order of the rules/lines does not appear to be relevant in any way, is that correct?

Also, if I use *~ instead of /**~, it seems I no longer need the "include the content of Samples~" rules. In my test repo, where I created various "~"-files, this seems to work now as intended. Or am I missing something?

So the complete ruleset to handle my original problem is now:

!^/(?!Library/|Assets/).*/Samples~$
!^/(?!Library/|Assets/).*/Documentation~$
/Library
*~
*~.meta

(this also continues to exclude Samples~ if it  appears in the /Assets subtree, as well as in the root folder, which is actually another requirement I needed)

 

3 hours ago, Héber said:

That's quite a special setup you have there!

True, but as Unity will convert their whole package system to their UPM format (including the Asset Store handling, from what I hear), this will be a situation all package developers will face in the near future.

Thanks again for your help! 😎

Cheers,

Wolfram

[Héber]

Hi Wolfram,

 

Quote

A question: The order of the rules/lines does not appear to be relevant in any way, is that correct?

Yes, you are right. The order doesn't affect at all. How do you feel about it? Would you prefer it to work having the order in mind?

Quote

Also, if I use *~ instead of /**~, it seems I no longer need the "include the content of Samples~" rules. In my test repo, where I created various "~"-files, this seems to work now as intended. Or am I missing something?

Just be aware that *~ is a name rule, if you use it, you can't deny rules inside the path of folders that ends with the ~symbol.

I'd say that it is not going to work when you want to deny things inside the Samples~ folder.

 

Best regards,

Héber.

 

[Wolfram]

On 1/29/2020 at 5:32 PM, Héber said:

Yes, you are right. The order doesn't affect at all. How do you feel about it? Would you prefer it to work having the order in mind?

Well, it would allow the user to create his own prioritization. For example, the problem I decsribed could have been immediately resolved by this (by stating, "do NOT ignore folders and their contents named "Samples~"", but then followed by "still, ignore everything in /Library, and everything (else) ending in ~").
But it might cause problems in other situations (and of course it would be a hassle for everyone modifying their ignore.conf to comply to this new behaviour), and even without it it was possible to resolve my problem.

On 1/29/2020 at 5:32 PM, Héber said:

Just be aware that *~ is a name rule, if you use it, you can't deny rules inside the path of folders that ends with the ~symbol.

I'd say that it is not going to work when you want to deny things inside the Samples~ folder.

Hm, this does not seem to be the case. With  the mentioned ruleset we now use, ~-Files in Samples~ are still ignored, and I am able to both explicitly ignore other files in Samples~, as well as un-ignore specific ~-Files in it.

[Héber]

Hi Wolfram,

 

Thanks for the feedback. We will have it in mind. However, as you said, changing it right now would annoy the rest of the users.

Quote

Hm, this does not seem to be the case. With  the mentioned ruleset we now use, ~-Files in Samples~ are still ignored, and I am able to both explicitly ignore other files in Samples~, as well as un-ignore specific ~-Files in it.

In that case, forget what I said.

Glad to hear you have it working as you wanted it.

 

Best regards,

Héber.