Merging in an xlink change will not automatically update the xlinked folder contents

[Mikael Kalms]

Hi,

when I change an xlink, I need to first check in the xlink change, and then click "update workspace" to make the contents of the xlinked folder update. I find the extra required step unintuitive, but accept that there are probably reasons for this. (Side question: Do I need to check in the xlink change? Or is it enough to change the xlink locally, and then click 'update workspace'?)

 

However -- if someone else merges from a branch with such an xlink change, they too need to perform an extra "Update workspace" to get the contents of their xlinked folder updated. This means that, sometimes, "merge latest from main to your task branch" is not enough to ensure that your workspace is up-to-date; sometimes you need to perform an additional "Update workspace" step to ensure that, and this depends on whether or not the merge introduced any xlink changes.

Am I missing something? If no, is there some way to make Plastic automatically update the xlinked folders within the workspace during such a merge?

[calbzam]

Hi,

- You are right that when you manually edit the Xlink target and checkin these changes, you will need to update the workspace to download the content from the new Xlink pointer. It could happen on a scenario where you are not performing the changes in the Xlinked repo via the parent repo so your Xlink is out of date.

- In the merge scenario: the merge is initially performed in the Xlinked repo and then in the parent repo. You will see that there are two merge links in your pending changes (one form the Xlinked and one for the parent repo). When you are running a merge, the new revisions created in the task branch will be downloaded to your workspace so when you you checkin the merge result, the Xlink should be pointing to the merge result changeset and your workspace should include all the revisions downloaded during the merge. An extra update shouldn't be necessary.

- If you are refering to some specific merge scenario, could you let me know the steps to reproduce?

Ideally, the Xlinks are designed to be modified always from the parent repo. This way, the Xlink target will be always pointing to the last and when you perform some changes in the Xlink, they will be automatically propagated to the XLinked repo.

Regards,

Carlos. 

[Mikael Kalms]

I'm not sure we are talking about the same thing. Allow me to elaborate a bit.

-----------------------

Fundamentally, everyone on our team expects Plastic to update the contents of an Xlinked folder at the same time that the XLink changes. Plastic doesn't work that way, and it causes frustration ("I merged from /main and now files are missing locally, yet /main is good for everyone else on the team") within our team.

Here are three scenarios where that is problematic for us when we use read-only XLinks. 

1) When a user creates or modifies an XLink, they need to check it in before they can validate that the xlink was set up correctly. The user needs to learn that the correct procedure is to create/modify the xlink, then check it in, then "Update Workspace". This is not great but is something that we can train people to do.

2) When a user merges, and the merge source includes an XLink creation/update, the user's workspace will not always get updated. This is problematic for the entire team: it means that when merging from /main to a task branch, then depending on what is in the merge source, the user may also need to perform an "Update Workspace" after the merge. In our case, it means that 1 out of 100 merges from /main to a task branch require the user to click "Update Workspace" afterward or else they get strange errors. They don't care that someone else has modified an XLink on /main, they want a merge from /main to their current task branch to provide them with an up-to-date workspace, always.

3) When a user merges from /main to a task branch, and the merge source includes an XLink deletion, _under some circumstances_ the XLinked folder will remain on-disk in the workspace. I don't know the exact conditions yet. In our case, it results in people occasionally re-adding the content from a deleted XLink (now as regular files rather than as XLinked files) because they don't always pay enough attention to exactly what they check in. ("Oh, I guess Unreal Editor also wanted to do something to those other files. I don't know why, but I trust UE knows what it is doing...")

-----------------------------

Below is a long 2-user journey, which demonstrate items 1 and 2.
 

-----------------------------

User A creates a repository, MainRepo, and a workspace. User A adds the file MainRepo-file1.txt to the repository, on /main.
<image 1>
User A creates another repository, SubRepo, and a workspace. User A adds the file SubRepo-file1.txt to the repository, on /main.
<image 2>
User A creates a feature branch, 'CreateXLink' in MainRepo.
<image 3>
User A adds a read-only XLink from MainRepo to SubRepo.
<image 4>
<image 5>
<image 6>

Above is the first situation that I find problematic: Since Plastic does not download the contents of the SubRepo changeset to MainRepo\SubRepo, there is no way for User A to validate that the configuration is correct without checking in results first. (For example, if MainRepo was a C++ application and SubRepo contained a library/plugin, User A cannot test-build the project on his/her local machine - the test-build would fail since no files are present in SubRepo yet.)

User A checks in the XLink add.
<image 7>
<image 8>
<image 9>

Above is the next situation that I find problematic: My local workspace is _not_ up-to-date with cs:2 -- the SubRepo folder is still empty -- yet there is nothing in the Plastic UI that indicates to me that my workspace is different from the repository. Nothing tells me that clicking "Update Workspace" will have any effect in this situation. Nothing tells me that switching workspace to cs:1, and back to cs:2, will make any difference to my workspace contents.

-----------------------------------

Now we introduce User B. User B will do some work in parallel with User A, directly on the main branch.

User B creates a workspace, MainRepo_2. User B switches to cs:1. User B adds the file MainRepo-file2.txt to the repository, on /main.
<image 10>
User B edits MainRepo-file2.txt but does not yet check in the results.
<image 11>
User A switches to /main and merges CreateXLink branch into /main.
<image 12>

User B notices that there are incoming changes, and clicks on the "view" button.
<image 13>
<image 14>

Here is the next problematic situation: User B is encouraged to use the 'Incoming changes' view, but is presented with a red error message. If user B ignores the red error message, goes back, and checks in the file, then this results in the file being marked as Checked-Out, and then the user is sent back to the red error message.
<image 15>
<image 16>

User B goes to the Branch Explorer view, performs a merge from cs:4, and checks in the result.
<image 17>
<image 18>
<image 19>

Here is the next problematic situation: User B's workspace is again out-of-sync with the repository contents: the MainRepo_2\SubRepo folder exists but is empty. User B's
workspace does not match the workspace contents, despite that all the user has done is to add a file and performed a merge when instructed to do so by the Plastic UI.
Again, if this was a C++ application with xlinks to libraries, then User B is now in a situation where fixing a one-line change in a source file, and then performing a merge as instructed by the Plastic UI, leads to compilation/linker errors on the user's machine until the user performs "Update Workspace" or switches the changeset back-and-forth.
<image 20>
<image 21>
<image 22>

User B edits MainRepo-file2.txt again, but does not yet check in the restuls.
<image 23>
<image 24>

User A now moves to cs:5, deletes the XLink locally (choosing to delete the items on disk), and checks in the change.
<image 25>

Next, slightly problematic situation: User B has learned from their mistake (the Incoming Changes red-text view) and merges immediately from cs:6. This results in a checkin window ON TOP OF the merge window. The users faithfully fills in a change description and then checks in the change. The user clicks 'update workspace'. After this, the user is returned to the original checkin window, and apparently the file didn't get checked in in the previous step?
<image 26>
<image 27>
<image 28>
<image 29>

User B attempts to check in the file again. This time, it succeeds.
<image 30>
<image 31>
 

[calbzam]

Hi,

For 1) and 2) I think at least a pop up explaining that after editing the Xlink the user may need to update their wrokspace could be helpful. Updating the workspace by default (without the user explicitly do it) I don't think is always a good idea as it could involve downloading tons of that and the user may prefer to do it on demand. But I agree that improving the user experience in this scenario would be very helpful.

Regarding 3), if you have steps to reproduce it, please let us know. Otherwise, we can setup a test in our labs based on your feedback to cehck if we can reproduce it and properly understand what's happening in this case.

Regards,

Carlos.

[Mikael Kalms]

I find it difficult to understand what sort of situations that a user would like to download on-the-other-side-of-xlink changes on demand.

I would understand if Plastic has a mode where the user can choose whether or not to download the contents of a folder (i.e. partial workspaces), and would use that to not download the xlink's contents at all.

I would understand if Plastic had a configuration where you could declare, for a workspace, which xlinks should be 'expanded' and which should not in the workspace. (This is partial workspaces, but at the granularity of xlinks instead of every possible folder.)

I find it far fetched, however, that a user would want to edit an xlink but keep the xlink's contents from a previous changeset within the workspace. What's the use case? The user effectively has an inconsistent local workspace at that point. The only use case I can imagine is "... so that the user can change the xlink, and check that in, without needing to download the contents" which I guess some CI processes would be interested in doing. Do you know of _any_ other use cases?

 

I understand if you don't want to change behavior because some existing customers rely on the current behavior. I find it hard to accept the current behavior as a good user experience, however.

 

For 1), a pop-up might help.

For 2), I'm not so sure. the user is not consciously updating the xlink. The user is merging in others' changes, which may or may not include xlink changes. When would the pop-up show? After every merge? After merges that include xlink modifications?

For 3), I'll report back with details in case I find a repro case in the future.

[Mikael Kalms]

Cycling back here. Just today, we had another instance of people being confused. "I merged from main and code isn't compiling for me. I had to check in the results of the merge, switch to another branch, and back again -- now, code is compiling again. What happened?"

People on my team do not understand that, if an incoming merge includes an xlink change, they need to check in the results of the merge and then click "Update Workspace" to get the xlinked files updated in their workspace. They do not understand that if a repository includes xlinks, the process for merging from main into a task branch is not the following:

1. Merge from main

2. Compile/test what's in the workspace. If it isn't working, undo & retry merge, or fix the problem directly in the workspace.

3. Check in results. DONE

-----

but rather, the following:

1. Merge from main

2. Review the list of changes included in the merge. If there are any xlink changes, go to step 5.

3. Compile/test what's in the workspace. If it isn't working, undo & retry merge, or fix the problem directly in the workspace.

4. Check in results. DONE

5. Check in results.

6. Click "Update Workspace".

7. Compile/test what's in the workspace If it isn't working, develop a fix and check in the fix in a second check-in -- or move the workspace back by one changeset, and delete the changeset created in step 5 -- or subtractive-merge the changeset created in step 5, and check that in, and go back to step 1 -- by the way, which method is preferable?

 

I am seriously considering giving up read-only xlinks for shared components, and manually copying/pasting those files into the parent repositories. The added confusion among the team -- particularly among content creators, who do not touch the code, they just want to merge latest from main, compile code, and run the application to edit their content -- is trending towards "not worth it". If we give up on read-only Xlinks because of perceived usability problems on our side, that is a sad day, both for us and you.

 

Here is a UserVoice suggestion on this: https://plasticscm.uservoice.com/forums/15467-general/suggestions/42508759-automatically-update-folder-contents-when-updating

[Mikael Kalms]

Here is a quote our internal Slack workspace, from earlier today. A programmer has made a major version update of the game engine, and as part of this, also updated a number of xlinks.

I have bolded one section, where I ask myself: Why does the programmer see the need to enlist the help of other programmers to help everyone on the team to correctly get latest? Why does he predict that people might come and ask "why is everything broken"? Why are people on our team using the PlasticSCM client incorrectly in this particular case? What can be done about it?

Quote

I started a validation build. If it succeeds I will make a post in the Slack that this is happening and tell people to update their workspace. I would appreciate if you programmers could help telling people that they need to update workspace in order for everything to work again (in case they come jumping in and wonder why everything is broken)

 

[Mikael Kalms]

This week we are once again running into a wave of these "I merged from main and now stuff isn't working on my local branch" moments of confusion from people on our team. Yes, we have updated an Xlink. Yes, we have reminded people to do "update workspace" after merging from main. No, everyone does not remember to do so.

[calbzam]

Hi Mikael,

Sorry for the delay. I think you had a meeting with my coworker Neha a few days ago and you reported the same concerns you mention in this thread.

I also had a meeting with Neha and the product team to find a good aproach to address this problem. 

Quote

For 1) and 2) I think at least a pop up explaining that after editing the Xlink the user may need to update their wrokspace could be helpful. Updating the workspace by default (without the user explicitly do it) I don't think is always a good idea as it could involve downloading tons of that and the user may prefer to do it on demand. But I agree that improving the user experience in this scenario would be very helpful.

I think some GUI indicator should help a bit and it would be easy to schedule a task for it. It should include also the scenario where changes in the Xlink target are performed via merge. We still haven't fully defined how this information would be shown in the GUI and your feedback is welcome.

We will keep you posted if any change is planned.

Sorry for the inconveniences,

Carlos.

[Mikael Kalms]

We have finally decided to stop using Xlinks, and instead include any plugins verbatim into our main repository.

The costs & confusion associated with Xlink updates, and difficult life cycle management (how do you know if a repo can be safely renamed/deleted? Perhaps it used to be xlinked by another repo a year ago?) finally made us say that it's not worth it for the upside of having separate change histories for each sub-repo.

[calbzam]

Hi Mikael,

Thanks for the update. Anyway, we are still considering your feedback regarding the Xlinks beahvior.

Sorry for the inconveniences,

Carlos.