Jump to content
Aki Kanerva

How do I fix an incorrectly detected move?

Recommended Posts

Hi there,

I'm dealing with a situation where there is a large number of files moved outside of Plastic SCM (Unity assets and their .meta files). For the majority of files, Plastic has detected the moves correctly. However, among the hundreds of files, a few have been detected incorrectly, such as .meta files for two assets have been swapped.

How do I tell Plastic that I want it to change the move it detected to point to a different pair of files?

Thanks!

-- Aki

Share this post


Link to post
Share on other sites

Hi Aki,

Please check "MOVE DETECTION" section in the following link:

https://www.plasticscm.com/documentation/gui/plastic-scm-version-control-gui-guide#Chapter4:ThePendingchangesVie

Quote

 

MOVE DETECTION
These options let you tune move detection. This is one of the key features in Plastic. Plastic can easily detect when you moved a file, renamed a directory, etc. But at the end of the day, it uses a heuristic and it can make mistakes. While it achieves very good results by default, these settings let you customize the heuristic to your needs.

 == Similarity percentage ==

Defines "how similar" two files need to be to detect them as moved or renamed. If you move foo.c to bar.c and modify it later, the percentage defines how similar the files need to be so that Plastic considers them the same file. If you changed a file a lot and you renamed it, and the file shows as added/deleted, chances are you need to tweak this setting so that Pending changes detect it as moved.
The similarity percentage applies to directories too. It defines how similar the directory structure needs to be—how many moved children relative to the total of directory entries.
Finally, it also applies to binaries. The percentage means the allowed difference in size.

 == Search matches ==

When a file under Plastic SCM is moved or renamed locally and then heavily changed, Plastic SCM may fail to recognize the item as moved and detects it as one added item and another deleted item. This is the case when the similarity percentage is below the threshold set for moved items detection (default 90%).
The search matches option lets the user match the added item with the deleted item to tell Plastic SCM that they are actually the same item and the history of the original item is preserved. It's the way to tell Plastic SCM that the items it is detecting as added + deleted are actually one single moved or renamed item.

Match added item with deleted to identify a move or rename
The similarity threshold can be adjusted for this specific item using the Min similarity accepted slider. As you move the slider to the left, the deleted items that match are shown in the list. Once the item is manually selected and the Accept the selected match button is clicked, the item appears in the "Moved items" category.

 

Regards,
Carlos.

Share this post


Link to post
Share on other sites

But I can't do that for a move that has already been detected. The "Search matches" item is greyed out for an item that Plastic has detected as moved.

Share this post


Link to post
Share on other sites

Hi,

What do you mean when the moved were incorrectly detected? I was guessing they were detected as add + delete instead of moved.

If we use controlled moves, Plastic won't need to match the moved files in disk. No sure if it helps. By the way, I can see you have opened a support thread to handle this issue and we asked for some more infomation to try to reproduce it.

Regards,

Carlos.

Share this post


Link to post
Share on other sites

Hi Carlos,

I'm referring to a situation where Plastic detects a move, but it's matching the wrong two files.

Here's a simplified example. Let's say I move two files:

Assets/Configs/A.txt --> Assets/Resources/A.txt
Assets/Configs/B.txt --> Assets/Resources/B.txt

But Plastic decides that they were swapped around:

Assets/Configs/A.txt --> Assets/Resources/B.txt
Assets/Configs/B.txt --> Assets/Resources/A.txt

This can happen often with Unity .meta files, because they have little content, so even small changes can significantly decrease the similarity percentage. Moving the files happened to coincide with Unity deciding to change their content unexpectedly. When you upgrade Unity to a new version that has new import settings, they don't get written into the .meta file until much later when the asset in question is reimported or has its import settings changed (which may not happen for months after the upgrade). Moving those files was related to a process that didn't change the assets, but Unity happened to change the .meta files by adding new default values.

It's not a big issue if there are just two files. However, in my scenario, there were 400 moved files. Among them were 6 wrong pairs of files. What I wanted to do was accept the other 394 move detections, and then manually fix the other 6. But Plastic doesn't let me do that - after a move has been detected, it can't be changed without turning off move detection entirely. I didn't want to manually match all 400 files.

In this scenario, changing the match percentage was of no help either. Increasing the threshold would have left out 200 files. (As an aside, all .meta file similarity percentages were 44% in this scenario.)

What cost me a lot of extra time was that after every single change in settings, or trying to manually perform a match, Plastic would forcibly auto-refresh itself and spend 5 minutes scanning the changed files. (I have to keep file content detection on because Unity often overwrites files with no changes to content.)

After 5 hours of struggling with this, I finally managed to think out of the box and realized I could split the moves into separate changesets. That solved the issue, but now the repository has a bunch of changesets instead of just one clean one.

Taking a hint from TortoiseHg, I would suggest a way to disable automatic detection of moves, and make that a user-initiated action instead. The user should be able to open a dialog that performs move detection, and then the user should be able to fine-tune the detected moves to their liking. Only after closing that dialog would Plastic need to rescan the working directory.

Also check out TortoiseHg's efficient context menu for choosing individual moves: when you right-click a file that isn't versioned, you get a context menu that lists all deleted files. Selecting one of these marks those files as a move pair. This is fast because it doesn't require any similarity matching (although I think the deleted files may be sorted by filename similarity to the right-clicked file - much faster than full content similarity checking. This would be great to have as an alternative to full content matching when Plastic is detecting moves.)

 

Regarding the support thread, I don't think I've done that. Maybe it's a different user with a similar issue?

Share this post


Link to post
Share on other sites

Hi,

- I'm afraid, you can only manually match the moved items if they were detected as add + delete (but not if they are already detected as moved). Not sure if by changing the similarity percentage, at some point the items as detected as add + delete and you can manually match them.

- Using controlled moves could also avoid the problem (moving the items via GUI or using "cm mv" command) but I understand this is not as comfoirtable as moving the items directly via Windows explorer.

- The "pending changes" view has an option to match moved files on when they have the same extension. Not sure if we can add a new setting to match moved files when they have the same name. I think it could help in your scenario.

Best regards,

Carlos.

Share this post


Link to post
Share on other sites
Quote

- I'm afraid, you can only manually match the moved items if they were detected as add + delete (but not if they are already detected as moved). Not sure if by changing the similarity percentage, at some point the items as detected as add + delete and you can manually match them.

That seems like a pretty big omission in the GUI, and a disadvantage over other clients like TortoiseHg. When you use an algorithm to do something automatically, it's a good idea to provide users the option of adjusting the results manually, without having to ignore the rest of the algorithm's results.

Quote

- Using controlled moves could also avoid the problem (moving the items via GUI or using "cm mv" command) but I understand this is not as comfoirtable as moving the items directly via Windows explorer.

It's less about the comfort, it's increased cognitive load that easily leads to human errors. There are already two different views into the same files (Explorer and Unity Project View). Adding a third one just makes things even more complicated for users. It's hard to remember what you're supposed to do and what to avoid in each view, and takes mental resources away from the actual development work.

It would be great if the Unity plugin could handle this. So when I move files in the Unity Project View, Plastic would detect that as a controlled move.

Quote

- The "pending changes" view has an option to match moved files on when they have the same extension. Not sure if we can add a new setting to match moved files when they have the same name. I think it could help in your scenario.

Yes, this is definitely a good idea!

Finally, I noticed that if I delete one folder in path A, and create a new folder in path B, Plastic will think that folder A has been moved to B - even if they have different names and paths. It would be great if I could right-click on a detected move and there would be a context menu item to "un-detect" that move.

Share this post


Link to post
Share on other sites

Aki, it looks like Carlos is having trouble understanding what you're asking for some reason.  It's possible and very simple to do exactly what you asked.  From Pending Changes, click "Options", then un-check "Find moved and renamed files and directories".  You can still use Search Matches to turn them into moves.  Here's a video.

Unity does have built-in support for Plastic SCM, such that it creates the proper moves from the moment of moving.  So does JetBrains Rider.  When I move files in either Unity or JetBrains Rider (with Plastic & Unity support turned on), they move the meta files on disk and via `cm`.

I generally don't bother moving files in the Workspace Explorer like we're supposed to because it's very inconvenient compared to other methods:  You have to use cut-and-paste because drag-and-drop is not supported.

Share this post


Link to post
Share on other sites

Hi @M-Pixel,

Yes, I know. My issue is that I have 400 files that were moved, but only 6 of them are matched incorrectly by Plastic. I really don't want to turn off "Find moved and renamed files and directories", because then I'd have to manually match the other 394 files.

Actually that was one of the first things I tried. But there were two more problems with that approach. First, when I turned off "Find moved and renamed files and directories", I no longer saw which files Plastic matched incorrectly. I had to turn it back on, then write down each mismatched file pair in Notepad, and then turn it off again. Then I used "Search Matches" to manually match the 6 offending files. But when I turned "Find moved and renamed files and directories" back on again, the manual matches I'd made had disappeared. Plastic's automatic matching was overriding the manual changes I'd made. So that turned out to be a dead end.

 

Thanks for mentioning that about the Unity plugin though. I have been using the built-in support all along, but for some reason moving files inside Unity wasn't detected that time. I tried it again, and now it works. No idea why it failed previously. (Although, to be fair, moving lots of files inside Unity is also somewhat inconvenient compared to Explorer.)

Share this post


Link to post
Share on other sites

Although you're past your issue, I figure I might as well mention this for anybody else that comes across this thread looking for a solution in a similar situation: in a scenario where you have hundreds of moves, the least-effort solution is probably to check-in the moves that were correct, then turn off auto-detect and check-in the other ones in a separate changeset.  While it might sound messy to have two commits where you could have one, ideally, these moves are being performed on a task-branch, so you would still have a single commit on your main-line.

Of course, this is still a work-around.  While I can understand from a software architecture perspective why Plastic does not allow you to selectively undo auto-moves (because the auto-moves are transient), it would be much better if Plastic SCM allowed auto-moves to be "staged" (or "locked-in"), just like how you can change items from Private to Added.

  • Like 1

Share this post


Link to post
Share on other sites
Quote

 

 I would suggest a way to disable automatic detection of moves, and make that a user-initiated action instead. The user should be able to open a dialog that performs move detection, and then the user should be able to fine-tune the detected moves to their liking. Only after closing that dialog would Plastic need to rescan the working directory.

Also check out TortoiseHg's efficient context menu for choosing individual moves: when you right-click a file that 

 

I shared this request with the team. We already internally commented some improvements about how the user can fine tune the move detections performed by Plastic. It would be a nicefeature but I'm afraid it's not scheduled for the near future.

Regards,

Carlos.

  • Like 1

Share this post


Link to post
Share on other sites

Hi all,

maybe you have a look at my user voice request 2 years ago suggesting that one can explicitly declare moves without move detection heuristics...

https://plasticscm.uservoice.com/forums/15467-general/suggestions/34800370-explicitly-tell-plastic-that-a-add-delete-is-in-fa
Feel free to upvote :) 

One could consider to extend this request in both directions (explicitly declare / undeclare moves).

Best
Jan

Share this post


Link to post
Share on other sites

I made a video showing the workflow in TortoiseHg Workbench. Here's a description of how it would have avoided the key pain points I had with Plastic SCM.

I've created a dummy project with some text files checked in. For demonstration purposes, the contents of three of these files are identical, to confuse automatic detection of renames.

  • Move files using Explorer to a different folder
  • Switch to TortoiseHg Workbench
  • Perform manual refresh to detect changes in the workspace
  • TortoiseHg now lists four removed files (in red, prefixed with "!") and four added files (in pink, prefixed with "?")

I can use two different methods for telling the SCM about renamed and moved files. First, I'm going to show the quick manual method, which has no equivalent in Plastic.

  • Right-click on an added file to bring up the context menu
  • Select "Was renamed from"
  • This brings up a submenu that lists all deleted files
  • I can now quickly pick the correct file from the submenu
  • No automatic move detection is performed at any point - this process would be fast even with thousands of files. Naturally finding the correct target from the submenu would be slow for the user, but I believe the submenu uses some kind of a quick sorting method, perhaps based on filename similarity, because I often find the correct files near the top of the list

The second method uses automatic move detection, similar to Plastic SCM. However, this is a user-initiated action on the workspace, rather than being a global preference that must be either on or off at all times - again reducing the amount of unnecessary processing compared to Plastic.

  • Select any number of added files
  • Right-click to bring up the context menu
  • Select "Detect Renames..."
  • This brings up a separate window for processing renames
  • Just like Plastic, I can now adjust the similarity percentage and then run contents comparison. However, unlike Plastic, I am able to adjust the percentage and run the detection multiple times. Each time, it will only run on the files selected in the left pane, rather than all files (which would be really slow).
  • Detection results are listed in the right pane
  • I can now select the matches I want, and manually accept them. If this was a real situation, some matches would have a higher percentage, and these would be prioritised, just like when Plastic chooses the best match. However, unlike Plastic, if the similarity percentage gives the wrong match (for example, a file was both renamed and changed, as happens often with Unity .meta files), I can still quickly fix it by manually choosing a different match. The only way to do this in Plastic is to globally turn off move detection, refresh the workspace, and manually try to find the correct files from among all changes (which are no longer easy to find due to the move detection being turned off).
  • I can keep making matches within this window until I'm satisfied, then close the window
  • Once the window is closed, TortoiseHg will refresh the workspace - in Plastic SCM, the workspace would have been unnecessarily refreshed several times, which can take minutes when there are hundreds of moved files.

Hope this helps clarify the workflow that is missing from Plastic.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...