How do I fix an incorrectly detected move?

[Aki Kanerva]

Hi there,

I'm dealing with a situation where there is a large number of files moved outside of Plastic SCM (Unity assets and their .meta files). For the majority of files, Plastic has detected the moves correctly. However, among the hundreds of files, a few have been detected incorrectly, such as .meta files for two assets have been swapped.

How do I tell Plastic that I want it to change the move it detected to point to a different pair of files?

Thanks!

-- Aki

[calbzam]

Hi Aki,

Please check "MOVE DETECTION" section in the following link:

https://www.plasticscm.com/documentation/gui/plastic-scm-version-control-gui-guide#Chapter4:ThePendingchangesVie

Quote

 

MOVE DETECTION
These options let you tune move detection. This is one of the key features in Plastic. Plastic can easily detect when you moved a file, renamed a directory, etc. But at the end of the day, it uses a heuristic and it can make mistakes. While it achieves very good results by default, these settings let you customize the heuristic to your needs.

 == Similarity percentage ==

Defines "how similar" two files need to be to detect them as moved or renamed. If you move foo.c to bar.c and modify it later, the percentage defines how similar the files need to be so that Plastic considers them the same file. If you changed a file a lot and you renamed it, and the file shows as added/deleted, chances are you need to tweak this setting so that Pending changes detect it as moved.
The similarity percentage applies to directories too. It defines how similar the directory structure needs to be—how many moved children relative to the total of directory entries.
Finally, it also applies to binaries. The percentage means the allowed difference in size.

 == Search matches ==

When a file under Plastic SCM is moved or renamed locally and then heavily changed, Plastic SCM may fail to recognize the item as moved and detects it as one added item and another deleted item. This is the case when the similarity percentage is below the threshold set for moved items detection (default 90%).
The search matches option lets the user match the added item with the deleted item to tell Plastic SCM that they are actually the same item and the history of the original item is preserved. It's the way to tell Plastic SCM that the items it is detecting as added + deleted are actually one single moved or renamed item.

The similarity threshold can be adjusted for this specific item using the Min similarity accepted slider. As you move the slider to the left, the deleted items that match are shown in the list. Once the item is manually selected and the Accept the selected match button is clicked, the item appears in the "Moved items" category.

 

Regards,

Carlos.

[Aki Kanerva]

But I can't do that for a move that has already been detected. The "Search matches" item is greyed out for an item that Plastic has detected as moved.

[calbzam]

Hi,

What do you mean when the moved were incorrectly detected? I was guessing they were detected as add + delete instead of moved.

If we use controlled moves, Plastic won't need to match the moved files in disk. No sure if it helps. By the way, I can see you have opened a support thread to handle this issue and we asked for some more infomation to try to reproduce it.

Regards,

Carlos.

[Aki Kanerva]

Hi Carlos,

I'm referring to a situation where Plastic detects a move, but it's matching the wrong two files.

Here's a simplified example. Let's say I move two files:

Assets/Configs/A.txt --> Assets/Resources/A.txt
Assets/Configs/B.txt --> Assets/Resources/B.txt

But Plastic decides that they were swapped around:

Assets/Configs/A.txt --> Assets/Resources/B.txt
Assets/Configs/B.txt --> Assets/Resources/A.txt

This can happen often with Unity .meta files, because they have little content, so even small changes can significantly decrease the similarity percentage. Moving the files happened to coincide with Unity deciding to change their content unexpectedly. When you upgrade Unity to a new version that has new import settings, they don't get written into the .meta file until much later when the asset in question is reimported or has its import settings changed (which may not happen for months after the upgrade). Moving those files was related to a process that didn't change the assets, but Unity happened to change the .meta files by adding new default values.

It's not a big issue if there are just two files. However, in my scenario, there were 400 moved files. Among them were 6 wrong pairs of files. What I wanted to do was accept the other 394 move detections, and then manually fix the other 6. But Plastic doesn't let me do that - after a move has been detected, it can't be changed without turning off move detection entirely. I didn't want to manually match all 400 files.

In this scenario, changing the match percentage was of no help either. Increasing the threshold would have left out 200 files. (As an aside, all .meta file similarity percentages were 44% in this scenario.)

What cost me a lot of extra time was that after every single change in settings, or trying to manually perform a match, Plastic would forcibly auto-refresh itself and spend 5 minutes scanning the changed files. (I have to keep file content detection on because Unity often overwrites files with no changes to content.)

After 5 hours of struggling with this, I finally managed to think out of the box and realized I could split the moves into separate changesets. That solved the issue, but now the repository has a bunch of changesets instead of just one clean one.

Taking a hint from TortoiseHg, I would suggest a way to disable automatic detection of moves, and make that a user-initiated action instead. The user should be able to open a dialog that performs move detection, and then the user should be able to fine-tune the detected moves to their liking. Only after closing that dialog would Plastic need to rescan the working directory.

Also check out TortoiseHg's efficient context menu for choosing individual moves: when you right-click a file that isn't versioned, you get a context menu that lists all deleted files. Selecting one of these marks those files as a move pair. This is fast because it doesn't require any similarity matching (although I think the deleted files may be sorted by filename similarity to the right-clicked file - much faster than full content similarity checking. This would be great to have as an alternative to full content matching when Plastic is detecting moves.)

 

Regarding the support thread, I don't think I've done that. Maybe it's a different user with a similar issue?

[calbzam]

Hi,

- I'm afraid, you can only manually match the moved items if they were detected as add + delete (but not if they are already detected as moved). Not sure if by changing the similarity percentage, at some point the items as detected as add + delete and you can manually match them.

- Using controlled moves could also avoid the problem (moving the items via GUI or using "cm mv" command) but I understand this is not as comfoirtable as moving the items directly via Windows explorer.

- The "pending changes" view has an option to match moved files on when they have the same extension. Not sure if we can add a new setting to match moved files when they have the same name. I think it could help in your scenario.

Best regards,

Carlos.

[Aki Kanerva]

Quote

- I'm afraid, you can only manually match the moved items if they were detected as add + delete (but not if they are already detected as moved). Not sure if by changing the similarity percentage, at some point the items as detected as add + delete and you can manually match them.

That seems like a pretty big omission in the GUI, and a disadvantage over other clients like TortoiseHg. When you use an algorithm to do something automatically, it's a good idea to provide users the option of adjusting the results manually, without having to ignore the rest of the algorithm's results.

Quote

- Using controlled moves could also avoid the problem (moving the items via GUI or using "cm mv" command) but I understand this is not as comfoirtable as moving the items directly via Windows explorer.

It's less about the comfort, it's increased cognitive load that easily leads to human errors. There are already two different views into the same files (Explorer and Unity Project View). Adding a third one just makes things even more complicated for users. It's hard to remember what you're supposed to do and what to avoid in each view, and takes mental resources away from the actual development work.

It would be great if the Unity plugin could handle this. So when I move files in the Unity Project View, Plastic would detect that as a controlled move.

Quote

- The "pending changes" view has an option to match moved files on when they have the same extension. Not sure if we can add a new setting to match moved files when they have the same name. I think it could help in your scenario.

Yes, this is definitely a good idea!

Finally, I noticed that if I delete one folder in path A, and create a new folder in path B, Plastic will think that folder A has been moved to B - even if they have different names and paths. It would be great if I could right-click on a detected move and there would be a context menu item to "un-detect" that move.

[M-Pixel]

Aki, it looks like Carlos is having trouble understanding what you're asking for some reason.  It's possible and very simple to do exactly what you asked.  From Pending Changes, click "Options", then un-check "Find moved and renamed files and directories".  You can still use Search Matches to turn them into moves.  Here's a video.

Unity does have built-in support for Plastic SCM, such that it creates the proper moves from the moment of moving.  So does JetBrains Rider.  When I move files in either Unity or JetBrains Rider (with Plastic & Unity support turned on), they move the meta files on disk and via `cm`.

I generally don't bother moving files in the Workspace Explorer like we're supposed to because it's very inconvenient compared to other methods:  You have to use cut-and-paste because drag-and-drop is not supported.

[Aki Kanerva]

Hi @M-Pixel,

Yes, I know. My issue is that I have 400 files that were moved, but only 6 of them are matched incorrectly by Plastic. I really don't want to turn off "Find moved and renamed files and directories", because then I'd have to manually match the other 394 files.

Actually that was one of the first things I tried. But there were two more problems with that approach. First, when I turned off "Find moved and renamed files and directories", I no longer saw which files Plastic matched incorrectly. I had to turn it back on, then write down each mismatched file pair in Notepad, and then turn it off again. Then I used "Search Matches" to manually match the 6 offending files. But when I turned "Find moved and renamed files and directories" back on again, the manual matches I'd made had disappeared. Plastic's automatic matching was overriding the manual changes I'd made. So that turned out to be a dead end.

 

Thanks for mentioning that about the Unity plugin though. I have been using the built-in support all along, but for some reason moving files inside Unity wasn't detected that time. I tried it again, and now it works. No idea why it failed previously. (Although, to be fair, moving lots of files inside Unity is also somewhat inconvenient compared to Explorer.)

[M-Pixel]

Although you're past your issue, I figure I might as well mention this for anybody else that comes across this thread looking for a solution in a similar situation: in a scenario where you have hundreds of moves, the least-effort solution is probably to check-in the moves that were correct, then turn off auto-detect and check-in the other ones in a separate changeset.  While it might sound messy to have two commits where you could have one, ideally, these moves are being performed on a task-branch, so you would still have a single commit on your main-line.

Of course, this is still a work-around.  While I can understand from a software architecture perspective why Plastic does not allow you to selectively undo auto-moves (because the auto-moves are transient), it would be much better if Plastic SCM allowed auto-moves to be "staged" (or "locked-in"), just like how you can change items from Private to Added.

[calbzam]

Quote

 

 I would suggest a way to disable automatic detection of moves, and make that a user-initiated action instead. The user should be able to open a dialog that performs move detection, and then the user should be able to fine-tune the detected moves to their liking. Only after closing that dialog would Plastic need to rescan the working directory.

Also check out TortoiseHg's efficient context menu for choosing individual moves: when you right-click a file that 

 

I shared this request with the team. We already internally commented some improvements about how the user can fine tune the move detections performed by Plastic. It would be a nicefeature but I'm afraid it's not scheduled for the near future.

Regards,

Carlos.

[JanReimer]

Hi all,

maybe you have a look at my user voice request 2 years ago suggesting that one can explicitly declare moves without move detection heuristics...

https://plasticscm.uservoice.com/forums/15467-general/suggestions/34800370-explicitly-tell-plastic-that-a-add-delete-is-in-fa
Feel free to upvote  

One could consider to extend this request in both directions (explicitly declare / undeclare moves).

Best
Jan

[Aki Kanerva]

Upvoted!

[Aki Kanerva]

I made a video showing the workflow in TortoiseHg Workbench. Here's a description of how it would have avoided the key pain points I had with Plastic SCM.

I've created a dummy project with some text files checked in. For demonstration purposes, the contents of three of these files are identical, to confuse automatic detection of renames.

• Move files using Explorer to a different folder
• Switch to TortoiseHg Workbench
• Perform manual refresh to detect changes in the workspace
• TortoiseHg now lists four removed files (in red, prefixed with "!") and four added files (in pink, prefixed with "?")

I can use two different methods for telling the SCM about renamed and moved files. First, I'm going to show the quick manual method, which has no equivalent in Plastic.

• Right-click on an added file to bring up the context menu
• Select "Was renamed from"
• This brings up a submenu that lists all deleted files
• I can now quickly pick the correct file from the submenu
• No automatic move detection is performed at any point - this process would be fast even with thousands of files. Naturally finding the correct target from the submenu would be slow for the user, but I believe the submenu uses some kind of a quick sorting method, perhaps based on filename similarity, because I often find the correct files near the top of the list

The second method uses automatic move detection, similar to Plastic SCM. However, this is a user-initiated action on the workspace, rather than being a global preference that must be either on or off at all times - again reducing the amount of unnecessary processing compared to Plastic.

• Select any number of added files
• Right-click to bring up the context menu
• Select "Detect Renames..."
• This brings up a separate window for processing renames
• Just like Plastic, I can now adjust the similarity percentage and then run contents comparison. However, unlike Plastic, I am able to adjust the percentage and run the detection multiple times. Each time, it will only run on the files selected in the left pane, rather than all files (which would be really slow).
• Detection results are listed in the right pane
• I can now select the matches I want, and manually accept them. If this was a real situation, some matches would have a higher percentage, and these would be prioritised, just like when Plastic chooses the best match. However, unlike Plastic, if the similarity percentage gives the wrong match (for example, a file was both renamed and changed, as happens often with Unity .meta files), I can still quickly fix it by manually choosing a different match. The only way to do this in Plastic is to globally turn off move detection, refresh the workspace, and manually try to find the correct files from among all changes (which are no longer easy to find due to the move detection being turned off).
• I can keep making matches within this window until I'm satisfied, then close the window
• Once the window is closed, TortoiseHg will refresh the workspace - in Plastic SCM, the workspace would have been unnecessarily refreshed several times, which can take minutes when there are hundreds of moved files.

Hope this helps clarify the workflow that is missing from Plastic.

[JanReimer]

@Aki Kanerva Nice summary of what this request is about! 👍

[Jashan Chittesh]

Sadly, this is still an issue in 2022 and I can't believe it. Plastic "matched" two files that have nothing to do with each other. One is a meta-file that Unity deleted because the asset no longer exists (it's a Unity package asset, not even a prefab). The other is the meta-file of a new prefab. Why don't I have a context-menu that tells Plastic "no, this is not a move"? This specific "move that wasn't a move" is particularly annoying because if I commit this, and someone else commits the deletion of the file that Unity deleted (which needs to be done), it will delete my prefab-meta file, breaking a lot of things really badly.

The thing that's really annoying about this is that when I turn off move detection, this changes ... nothing. It still shows this nonsense move that is no move. Even when I disable move detection, close Plastic, re-open Plastic (and of course, I have tried refreshing a few times):

I had even tried re-adding the old meta-file. But then, Plastic thinks "oh, there's a new file". So it looks like what I need to do is make a safe copy of my new prefab meta-file, undo the change, restore the safe copy (yeah, that did "solve" this for me, this time). This is not the first time Plastic destroys productivity for us.

I have tried both, the legacy Plastic and the alpha. Same issue. I'm a few versions behind (11.0.16.6787) and looks like there's no "check for updates" button anywhere so unless I clicked on the tip when it showed up, I need to download and re-install via the Website. But even after that: Same issue with the current version (11.0.16.6949).

[calbzam]

Hi @Jashan Chittesh,

Can you check if Plastic is detecting the change a "Moved" or "Locally moved"?

The locally moved items are the changes that Plastic is trying to match on disk. You will need to enable the move detection in the "Pending Changes" view.

If the file is detected as controlled moved, it means the moved happened under Plastic control (via Plastic GUI, Unity plugin, "cm move" command...).

For the moved items, Plastic doesn't need to match any file on disk to determine if this is a move or not because it already has the information of a moved that happened in the workspace.

Regards,

Carlos.

[juanitogan]

Wow, this is severe.  I just wasted 3 hours of my life trying to correct a misunderstood move by Plastic.  As far as I can tell, this "feature" doesn't work much as advertised.

I renamed a .cs.BAK file back to .cs and edited it a bit after renaming it.  Plastic picked up the .cs.BAK.meta to .cs.meta file move okay.  But the .cs file itself was picked up as a delete/add.  When trying to use Search Matches on the added file, it finds nothing even though it should easily be at least an 80% match.  Even at 10%, nothing.  Then, at 0%, it found the match... or so I mistakenly thought.  I clicked Okay...... but it had really matched the .cs.BAK.meta file to the .cs file.  Ack!  The field was too small to notice the .meta before I clicked it.  Ugh.  Why oh why can't you see **all** deleted files at 0%!!!!!!

And it just gets worse from there.  Once that mismatch was locked in, I could find no way to undo it properly.  I tried turning off move detection and all sorts of other goofy things.  I even tried the `cm mv` CLI command on the .cs and .meta files and nothing would break that bad move bond.  Tried renaming back to BAK.  Tried deleting everything, restarting, restoring.  Tried both sets of files (BAK and non-BAK).  Finally, I gave in trying to do it the right way, backup up my changes to elsewhere, rolled back changes on the .cs file, and restored manually from the BAK files point.  Then used `cm mv` to force the rename and update correctly.  Yikes!

This is seriously bad that the above complaints are here and this is still the funky feature/quagmire that it is.  I didn't like the force move from the stupidly-simple Collab to Plastic from day one -- especially, considering how often Plastic crashes.  Also, just to remind me regularly of the pain of Plastic, the extreme extra padding around the Plastic icons in the project tree in Unity 2017 is annoying and sloppy.  Sure, I see some of the potential here with Plastic but, boy, does it fall short.  I use git on most of my projects, like it somewhat even though it is overly complex for most users, but really liked the nice niche Collab filled on small projects.  I've wasted 1.5 days just moving to Plastic on this project (1 day on the main Windows platform, .5 days to get it working on the test Mac), and now another 3 hours gone.  How much more pain is yet to come?

Finally, that percentage slider in the Move Detection tab of the Options window keeps ticking down by 1% every time I go in there and hit OK.  What's up with that slop?

And, oh yeah, why do 3 untouched prefabs in the same folder keep popping up as changed between commits even though the diffs always come out clean???????  I keep committing them, for lack of a better option, and they keep coming back (and their histories do not update).  This really is a what-not tool.

[calbzam]

Hi @juanitogan, you mention different issues in this thread. If you open a  ticket at support@plasticscm.com, we will be happy to arrange a meeting to with to fully understand all your problems (the issue manually matching the moves, the problems you are facing after switching from Collab and this issue where some files keep appearing as changed, crashes...).