Recommended workflow / project structure for project with code & art

[scottrt]

We're working on a Unity project that will have lots of large binary files (art). I'd like to have a way to backup and share the art source as well as manage code. What's a recommended workflow for this? I was thinking of something like the following:

• ProjectRepo
  • Unity
  • ArtSource

I'm not even sure whether artists use version control very much, but the art is somewhat critical for our project, and will be operated on by people in separate locations, so it's important that it's backed up and shared easily.

I like the ability in Plastic to just download the pieces you want to work on. This way coders wouldn't need to pull gigabytes of art, and artists wouldn't need to pull the whole repo just to make a small change.

One downside I see is that our repo could quickly get out of hand if we change a large binary 20 times. 

Apologies if there's an guide for this--I did some forum searching and skimming through the plastic book, but didn't see anything obvious. I'm very new to Plastic, but have used git for a number of years, and never had a good workflow for artists beyond checking in FBXes into git, and trying to remember to backup versions of the source art into Google Drive or something.

 

[calbzam]

Hi,

- Are you aware of Gluon? The version control for artsits? It allows to easily configure the workspace just downloading the files the artist need and locking them before modifying. It eases a lot the workflow for artsits:

https://www.plasticscm.com/gluon

- Plastic has the possibility to use XLinks if you want to have separate repos in your project:

https://www.plasticscm.com/documentation/xlinks/plastic-scm-version-control-xlinks-guide

- We have a feature request (that should be scheduled soon) that helps reducing the size of the databases:

 

Regards,

Carlos.

[scottrt]

Thanks, yeah I was looking at Gluon. The trimming feature sounds very useful, especially if you accidentally commit a  few 1 GB files.

Overall, just curious how other people organize similar projects. Everything in one repo? Separate art repo? Ad-hoc backups for art source?

[Mikael Kalms]

If you are happy to just use the features from Gluon (so you will not do any branching/merging) then you can put all your stuff into a single repo, just like you speculate at the beginning.

We made a turn-based strategy game in Unity. We really wanted to use task branches for the Unity project itself, but were concerned about repo size & workspace size, just like you. For that project, we kept all source assets in Google Drive, and the Unity project in Plastic. We used the full Plastic SCM client (not Gluon).

Google Drive provided a simple way of sharing files. It provided a backup in case of catastrophic disk failure.

The Plastic repo ended up at 85GB including history.

Overall, this worked well for us. The main problems when using Google Drive for source assets are, 1) it is not obvious to users when synchronization is complete, and 2) it is cumbersome to look at history, 3) there is no linkage between "source asset version X" and "imported asset version X".

 

We are now making larger FPS in Unreal. We still want to use task branches for the Unreal project. Our source asset collection is so large that if we were to put it in Plastic, we don't want to fetch all of it when getting latest. For this project, we keep all source assets in one Plastic repo, and the Unreal project in another repo. We use Gluon (with partial checkout) for the former repo, and Plastic SCM (using branches) for the latter repo.

The source assets repo is currently at 650GB including history. A full checkout would be >100GB, but no-one does that.

The Unreal project repo is currently at 300GB including history. A checkout is ~25GB.

Overall, this works well for us. The main problem when using two separate repos are, there is no linkage between "source asset version X" and "imported asset version X", other than looking at timestamps. It does not cause us much trouble in practice.

 

We are evaluating the Dynamic Workspaces feature. In the future, it may allow us to combine both repos into one, while still allowing us to use branch-based workflows, yet only requiring people to download the files that they access. It is not flexible enough for us yet; we need it to work on multiple OSes and possibly in Docker containers to satisfy our build system needs.

[scottrt]

That's incredibly helpful Mikael, thanks for all the insights. I may be conflating the need for a good backup system with the need to go back to previous versions of art. It's interesting that you don't have a lot of trouble associating art versions with code versions.

[Mikael Kalms]

Today, we lack linkage between source art versions and imported art versions.

We have strong linkage (files in same repo) between imported art versions and code.

If we didn't have the latter linkage, we would be in a world of pain

[kpaxart]

On 9/18/2021 at 6:24 PM, Mikael Kalms said:

If you are happy to just use the features from Gluon (so you will not do any branching/merging) then you can put all your stuff into a single repo, just like you speculate at the beginning.

We made a turn-based strategy game in Unity. We really wanted to use task branches for the Unity project itself, but were concerned about repo size & workspace size, just like you. For that project, we kept all source assets in Google Drive, and the Unity project in Plastic. We used the full Plastic SCM client (not Gluon).

Google Drive provided a simple way of sharing files. It provided a backup in case of catastrophic disk failure.

The Plastic repo ended up at 85GB including history.

Overall, this worked well for us. The main problems when using Google Drive for source assets are, 1) it is not obvious to users when synchronization is complete, and 2) it is cumbersome to look at history, 3) there is no linkage between "source asset version X" and "imported asset version X".

 

We are now making larger FPS in Unreal. We still want to use task branches for the Unreal project. Our source asset collection is so large that if we were to put it in Plastic, we don't want to fetch all of it when getting latest. For this project, we keep all source assets in one Plastic repo, and the Unreal project in another repo. We use Gluon (with partial checkout) for the former repo, and Plastic SCM (using branches) for the latter repo.

The source assets repo is currently at 650GB including history. A full checkout would be >100GB, but no-one does that.

The Unreal project repo is currently at 300GB including history. A checkout is ~25GB.

Overall, this works well for us. The main problem when using two separate repos are, there is no linkage between "source asset version X" and "imported asset version X", other than looking at timestamps. It does not cause us much trouble in practice.

 

We are evaluating the Dynamic Workspaces feature. In the future, it may allow us to combine both repos into one, while still allowing us to use branch-based workflows, yet only requiring people to download the files that they access. It is not flexible enough for us yet; we need it to work on multiple OSes and possibly in Docker containers to satisfy our build system needs.

so you separate the artist repo and program repo , the artist uses the gluon and programmer use the plastic, the artist repo without any code. did I understand right? thank you.