Package from Git URL shows LFS Pointer file instead of actual content

Hello everyone, I have a problem with a custom package that we’re working on for our customer. I’ve studied the documentation thoroughly but I cannot seem to find a solution.

Our setup consists of two Unity Projects that use Unity 2020.1.4f1:

Project A is the project that we work on the package in <Unity project folder>/Packages/com.mycompanyname.something/ and is hosted on Github.

Project B expects this package in it’s manifest.json

{
    "dependencies": {
            "com.mycompanyname.something": "https://github.com/mycompanyname/reponame.git?path=/Packages/com.mycompanyname.something"
    }
}

While we do not have any problems using Git LFS while pulling/pushing Project A, Project B loads the package but all files tracked with LFS just show the pointer files and not the actual files.

This suggests to me that Git LFS is not triggered by the way the package manager pulls packages from Github.

This command in bash tells me I have only one git-lfs.exe on my pc (/c/Program Files/Git LFS/git-lfs):
where git-lfs

This tells me that it is also using this exact file(/c/Program Files/Git LFS/git-lfs):
which git-lfs

This tells me that GIT_LFS_PATH=C:\Program Files\Git LFS is correct:
git-lfs env

I attached an extract from the Editor.log where it resolves the packages. Nothing interesting to see here.

Could this have something to do with administrative permissions or any other windows permissions in general?

I wonder, why is Project B unable to resolve the files tracked by lfs and instead loads the lfs pointer files?

6449511–722142–Editor.log.txt (14.5 KB)

2 Likes

Hello,

There was an issue where the package manager failed to download files from git repository when the path was present, this seems to be affecting you on Project B. In the issue tracker you can find the information of the Unity build where this is fixed, please let us know if upgrading your project to some of these versions doesn’t fix your problem for Project B.

Thank you for your quick reply! After upgrading to 2020.1.10f1 I get a different error. After a fixed duration of about 2 minutes of resolving packages, I automatically get this:

It seems to me that it now starts resolving via Git LFS but does not wait for it to be finished but instead after 2 minutes prompts this error (I have a slow internet connection).

Edit: It seems my problem is this: Unity Issue Tracker - Unity fails to open project with error: Failed to resolve packages: An operation that requires exclusive access to the project..

Hello again, I’m so sorry you are hitting this 2nd issue. We are actively working on a solution for this timeout problem, we found already the source of this problem and a fix will be landing soon.

I’m hitting this issue in 2020.3.14, and in the comments on that issue track you’ll see someone with the issue in the latest LTS version as well, time of writing is 2020.3.16. I’ve tried enabling git lfs cache in upm user config file on macos, it’s not changing the outcome. My files that are stoed in lfs are showing a few lines git lfs related text instead of their proper contents.

Hey, I’m really sorry you are having this issue, I’m not sure if the source of the problem is the same that the originally fixed one, it would be super helpful if you can submit a bug with all the possible information about it. The page detailing the bug reporting process can be found here: https://unity3d.com/unity/qa/bug-reporting.

If you could change the Package Manager Log Level to “Silly” before submitting the bug. This option can be found under Preferences, in the General tab.
7466192--916913--upload_2021-9-1_14-15-6.jpg

You can share the case number here once you submit the bug and I can follow up on it.

Thanks in advance!

I had a similar error about corrupt PNGs from a package. I fixed it reinstalling git lfs:
brew install git-lfs

For anyone arriving here, the workaround (still) is to copy/move your .gitattributes to the same folder as package.json, as per: Unity Issue Tracker - Package Manager fails to download LFS files from Git repository when the Git URL contains a path extension

Confirmed in unity 2021.3.18f1 (latest as of today)

Hi @thefallengamesstudio , that bug was fixed a while ago. If I understand correctly, you are still running into that issue in a more recent version of the Editor? (2021.3.18f1) If so, can you provide repro steps (maybe a specific small Git repository containing a dummy package we can use in a repro) so that we can track that regression and address it?

Hey! Yes, correct. Where should I send it?

Meanwhile, the particularities are:

  • package’s master branch is a normal Unity project with a “Assets/upm” folder, and a .gitattributes at the root
  • on release, I use git subtree to create a vX.Y.Z-upm branch that only contains the “Assets/upm” directory’s contents
  • on the consumer side, the package is declared as
    "com.<company>.<product>": "https://<token>@github.com/<user>/<repo>.git#v0.5.4-upm"

I have a feeling that just going over this list could give you a hint as to why it’s not working, because it’s an otherwise regular package, just some images and scripts

What repo is Git subtree used with? I assume it’s a different repo intended for releases of your package? Does the vX.Y.Z-upm branch also contain the .gitattributes file? It has to be present in every branch that uses Git LFS files, not only the main/master branch.

1 Like

Nope, same repo. Reason is: have a branch with just that Assets/upm/* content at the root, to not need to add the path argument into the url.

But you make a very good point I oversaw… with my setup, there won’t be any .gitattributes in the resulted branch, since that subfolder’s content replaces everything in the repo now.
Good point! Hope this helps others that are in the same situation

If you do this via a script, all you need to add is something like this after creating the substree:

git checkout master -- .gitattributes # maybe other "repo root" files, if any are needed
git add .gitattributes
git commit -m "Add Git repo files to release"

BTW, your setup is pretty clever, I never thought of using subtrees in the same repo to “move” things around. That’s a very interesting approach. It has some gotchas though; aside from .gitattributes, there may other files you want to add, like .gitignore (or maybe not, since your branch is actually more or less immutable branch… in which case, tags could be an option too, if you want to avoid polluting your list of branches).

So just to be sure, this missing .gitattributes file in your branch is what was causing the issue, right?

1 Like

Faced the same problem, and same solution; copying .gitattributes into the root of the UPM package created by git subtree split fixed the issue with git lfs not fetching the lfs file contents. Thanks.

1 Like

Sorry, not sure what happened when I tried to reply at the time, but yes, you’re correct. Copying that file fixes it!

1 Like

Hi !

I’m facing this issue right now. I add a package from a private GitHub repo and LFS files are not resolved, only the pointer is pulled.

I install the package with a specific branch, in the URL (ex : https://github.com/MyOrg/MyRepo.git#develop). The .gitattributes file is at the root of the package, in the same folder as the package.json file. And I have Git LFS installed on my PC (Windows 11).

None of the solution in this thread or in the documentation works …

3 Likes

@anon30979860 I am also still seeing this issue in Unity 6 (6000.0.24f1).

I have a UPM package at a git URL containing both a path and a branch (#). That git repo uses LFS (all DLL/PDB files are tracked with LFS), and I have verified that a .gitattributes file containing those tracking rules exists on the referenced branch in the URL. However, when I install that package, the installed DLL/PDB files (under Library/PackageCache/<MyPackage>/Runtime) are just 1KB LFS pointer files, not the expected ~70KB binary files. I have experienced this in two different projects referencing the same UPM package. Copying the .gitattributes file to the referenced path also does not help. Is this a reversion of the aforementioned bug perhaps?

On further investigation, it looks like I needed two things to solve my LFS issue:

  1. The .gitattributes file does need to be in the referenced folder path; it can’t just be at the root of the repo, per comments on that bug. That means, if there is more than one UPM package hosted in the same git repo, then each package needs a copy of the .gitattributes file. Symlinking won’t work either; git can store symlink files but apparently it doesn’t follow symlinks for .gitattributes files.
  2. The .gitattributes file within these paths cannot define macro attributes. I had defined an lfs macro attribute ([attr]lfs filter=lfs diff=lfs merge=lfs -text), but apparently macro attributes may only be defined in a top-level .gitattributes file (i.e., at the root of the repo). This is a restriction of git, not Unity.

Also for anybody dealing with UPM/LFS issues like this in the future, one quick way that I’ve found to test if the Unity Package Manager is correctly pulling LFS files is just to look at the file size. If you’ve got a DLL or PNG file that’s only 1KB, then it’s probably just an LFS pointer file, not the actual binary that you intended (you can further verify by opening the file in a text editor).

@anon30979860 @supita_unity I’m still unclear why we need a .gitattributes file inside each package. It would be really nice if the Unity Package Manager would scan up the package’s parent directories for .gitattributes files so that we could use a single root one when hosting multiple packages in a single repo.