Is it good practice to revert unchanged items?

[phvalve]

Unity often touches various assets which marks the file as changed even when the file remains virtually the same.

Should I manually run the revert unchanged items or does Gluon check the file contents during check-in and omit identical files from getting to the repo?

On that  same note, does Gluon ( orPlastic) do any kind of delta encoding with binary files? Often the repository size in cloud is smaller than the local project and this is something I've been wondering for a while.

Thanks for reading!

[calbzam]

Hi,

During the checkin operation, a new file revision is not created if the file content keeps the same as before. The "Checkin changes" view can be also re-configured so it checks the file hash to determine if a file has changed (but this calculation takes longer time than just checking the timestamp).

When you create a new file revision, it's stored in the database (not using deltas). But note that creating new branches (or even merging them) doesn't mean that the repo size will increase. It will only increase if these operations involve new file revisions.

If the repo size is not the same, it could mean that you haven't replicated all the branches.

Regards,

Carlos.

 

[David Yang]

Quote

Often the repository size in cloud is smaller than the local project

It might also have something to do with compression? Or because cloud is currently using a cloud-native database rather than Jet database.

 

Quote

When you create a new file revision, it's stored in the database (not using deltas).

@calbzam or @psantosl : Quick question - any reason why Plastic stores the whole file as a revision and not just the deltas? Is it because many binary files are saved with compression so storing deltas won't save space? What about binary files saved without compression though - wouldn't storing deltas make a big difference to repo size over time?

Thanks

David

 

[phvalve]

@calbzam So to summarise, it is not necessary to revert unchanged items because Plastic catches those during checkin? Just making sure.

I'm not worried about the repo size too much. Compression would explain the difference.

[calbzam]

That's right. Plastic won't create new revisions for those items. And if don't even want to see them in the "Pending changes" view, you can also reconfigure it as I explained.

Regards,

Carlos.

[phvalve]

Thanks for answering my questions @calbzam, I only have one more.

I noticed I asked about deltas in binary files but what about text assets? Unity stores many files in readable YAML format. If one line in a 5mb plain text file is updated, does this cause a new 5mb file revision or is the change stored in a more fine grained fashion? Does it make a difference if the plain text file is 50mb? (Asking for a friend)

[calbzam]

Hi,

The revisions are compressed in the database but as we commented, every new revision is stored in the database (not using deltas). Also for the text files.

Regards,

Carlos.

[psantosl]

Hi,

 

Quote

Quick question - any reason why Plastic stores the whole file as a revision and not just the deltas? Is it because many binary files are saved with compression so storing deltas won't save space? What about binary files saved without compression though - wouldn't storing deltas make a big difference to repo size over time?

 

Yes, we do it for performance reasons. Not having to rebuild deltas is much, much faster.

 

We dabbled with the idea of doing deltas for a while, and we'd like to introduce Xdeltas for big binaries. But for text files (code, etc), compressing works quite well and reconstructing the revision is linear time.

 

We actually took this decision back in 2005 when we made the first tests to compare compression vs deltas, and we revisited the idea multiple times since, but as I said, we decided to stick with compression.

[David Yang]

Thanks Pablo, that seems fair.

At the time of asking, I was playing around with a free-tier Windows EC2 instance (and Windows itself took up most of the available storage), so was very conscious of space! But yes, can understand why this would be much better for performance, and good trade-off when compression works so well for text files.

For big binaries, perhaps a hybrid approach could work? Eg, store a full revision every X revisions, and deltas in between? That way it would limit the amount of tree-walking / delta rebuilding needed, but still save up to X revisions worth of space...

[psantosl]

Exactly, all delta implementations I'm aware of implement some sort of window schema, so you don't have to walk back a huge history.

 

That's something we have in mind, and we did quite a good amount of research and study of Xdelta to apply it to binaries, but it is not in the top of the prio list, I'm afraid. 🙂