Question about perf when invoking status during an asset move.

[DeadMoney]

With the UE plugin enabled and using Cloud edition, an asset move invokes this kind of command chain. You'll notice the status command takes a fairly long amount of time. I figure this is because it's doing a directory status on the entire Content/Marketplace folder. Is this guess correct? Would it be a performance improvement to status only the destination file on copy? Or is the status command here for some other purpose?

This would probably also save Plastic some bandwidth if these status queries are cloud-live.

Another guess is that due to me having a very large open changeset that's slowing things down?

Anyway, if I knew better what caused this I could make sure to change my workflow in the future.

Log          LogSourceControl          ExecuteSynchronousCommand: Copy
Log          LogSourceControl          Moving C:/dev/dead-money/maki/Maki/Content/Marketplace/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset to C:/dev/dead-money/maki/Maki/Content/Marketplace/Environment/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset...
Log          LogSourceControl          RunCommand: 'undochange "C:/dev/dead-money/maki/Maki/Content/Marketplace/Environment/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset"' (in 0.152s) output (175 chars):
Log          LogSourceControl          c:\dev\dead-money\maki\Maki\Content\Marketplace\Environment\PolygonDungeonRealms\Meshes\Characters\SK_Chr_Hero_Female_01_Skeleton.uasset unchecked out correctly
Log          LogSourceControl          RunCommand: 'move --nomoveondisk "C:/dev/dead-money/maki/Maki/Content/Marketplace/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset" "C:/dev/dead-money/maki/Maki/Content/Marketplace/Environment/PolygonDungeonRealms/M' (in 0.156s) (output 307 chars not displayed)
Log          LogSourceControl          RunCommand: 'add "C:/dev/dead-money/maki/Maki/Content/Marketplace/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset"' (in 0.324s) (output 223 chars not displayed)
Log          LogSourceControl          RunCommand: 'status --compact --noheaders --all --ignored "C:/dev/dead-money/maki/Maki/Content/Marketplace/"' (in 1.857s) (output 2595691 chars not displayed)
Log          SourceControl             The selected items are about to be added. Please wait ...
Log          SourceControl             Item c:\dev\dead-money\maki\Maki\Content\Marketplace\PolygonDungeonRealms\Meshes\Characters\SK_Chr_Hero_Female_01_Skeleton.uasset was correctly added
Log          LogSourceControl          Copy of 1 files processed in 2.523s

[DeadMoney]

More context: in this case I'm just doing some cleanup, moving some assets from Content/Marketplace/[asset path] to Content/Marketplace/Environment/[asset path]

These synty packs have 1000s of files, so if someone in Cloud SCM ops gets a pager call ...

[SRombauts]

Hello, thanks for reaching out with the logs, it's always very helpful!

Can you please just share the versions of Unreal Engine & the plugin you are using? I can already see from the logs it's pretty recent so this is just for extra clarity.

You are right in your analysis, what is slowing your workflow a lot is the directory status on the entire Content/Marketplace! It's a side effect of another improvement required by Unreal Engine 5, that group all status queries together with their common directory, which in your example is

Quote

Maki/Content/Marketplace/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset
Maki/Content/Marketplace/Environment/PolygonDungeonRealms/Meshes/Characters/SK_Chr_Hero_Female_01_Skeleton.uasset

The optimization was made after a series of workaround to deal on how both Unreal & Plastic SCM works, to avoid making many individual calls, or making class with too many files listed

Unreal does require to update the status of both the source & the destination (to correctly confirm it as a rename) but it would certainly be worth adding a specific use case for updating the status of only 2 files!

Would you be able to give a try to this little change I made to test this scenario in a dedicated branch?
https://github.com/SRombauts/UEPlasticPlugin/commits/improve-move-file-performance

Cheers!
Sébastien

[DeadMoney]

I'm using Unreal 5.1 and PlasticSCM plugin 1.6.2

I'll set up an experiment for you this week.

[SRombauts]

Hello, thank you.

I should have added that I made some experiments myself on the biggest project I have at hand, with around 100k assets. The results are;

1. In case of a move from what folder to another, the change I made reduce the time by ~2.918s as part of what was a 3.877s Copy operation!
2. In case a the simple rename of one file in one folder, the change increase the time by ~100ms as part of a ~1.1s rename operation

This was just a testing branch, and I have ideas on how to make the change hopefully better by avoiding increasing the time in the scenario 2.

[SRombauts]

For the record, I have been working on this subject recently; I will run more benchmarks and try to come up with a solution that does not only improve perf on big project, but also improve timing or at least don't degrade them on smaller project!