Upload sources cache ignores Crowdin branches

[finebel]

Currently, the Crowdin cache is structured like this:

{
  "sourceHashes": {
    <file path>: <hash>,
    <file path>: <hash>,
    ...
  }
}

The file paths don't include any (Crowdin) branch information. This results in the following behaviour:

Example 1

1. Run crowdin upload sources --branch branch1 --cache
2. Check out a different Git branch and run crowdin upload sources --branch branch2 --cache

No sources are uploaded for 2 (because file path and hash didn't change).

Example 2

1. Delete "string 1" from foo.strings
2. Run crowdin upload sources --branch branch1 --cache
3. Checkout another Git branch
4. Delete "string 1" from foo.strings (now, foo.strings has the same content as in 1)
5. Run crowdin upload sources --branch branch2 --cache

Changes aren't reflected in Crowdin branch2 because in 5, no sources are uploaded (file path and hash didn't change).

---

In the context of CI, you can circumvent the problem by including the (Git) branch name in the caching key as outlined here.
Wouldn't it make sense to include the Crowdin branch information directly in the cache itself? Or are there specific reasons why you decided against this?

[andrii-bodnar]

@finebel thanks for the feedback!

We considered including the branch name into the local cache key, but didn't find enough reasoning for it. Currently, the cache depends only on the file content and when it changes, we upload the file to Crowdin.

In most cases, only a small portion of files differs between branches. Therefore, it's more effective to reuse the content hash to avoid unnecessary uploads when switching branches.

However, your use case is quite interesting and makes sense. Regarding the Example 1, it should be fixed within the #970.

[finebel]

Therefore, it's more effective to reuse the content hash to avoid unnecessary uploads when switching branches.

I personally would argue, that correctness is more important than convenience but I guess that's a prodcut decission you have to make.

Maybe you could also use a (probably more complex) different approach where you store the hash sums on the server. When uploading sources, you would download them (all at once) from the server and then you would use these downloaded hash sums to decide whether a new upload is necessary or not. The local hash would then become obsolete.
This would have also the advantage that potential unnecessary initial uploads on new machines are avoided.

[andrii-bodnar]

Thanks for the feedback, @finebel! The --cache feature is still experimental, so your feedback will definitely help us shape it properly. We'll take this into account. Ideally, we would like to avoid making any changes to the platform and only touch the CLI side.