Git support on Package Manager

[rz_0lento]

2018.3.0b7 added following:

• Package Manager: Add experimental support for Git package dependencies.

• Package Manager: Add experimental support for scoped registries (private registries)

but docs haven’t been updated for this yet, would it be possible to get example what to manually put into manifest to ref specific git branch?

[rz_0lento]

Figured it out, syntax is:
“PackageName”: “Loading...”,

or if you need specific branch, use:
“PackageName”: “Loading...”,

[okcompute_unity]

Hi @rz_0lento ,

Glad you have found the pattern :). As stated in the release notes, this is an experimental feature. We are targeting production ready quality for 2019.1. Super glad that you are trying this out though. Please, let us know if you find any issues.

And here is a minimalist documentation if you would like to expand your testing further and maybe for others who are reading this thread.

Requirements

To use Git packages in a project, Git must be installed on the user machine. The Git executable path should be listed in the PATH system environment variable.

Project manifest lock attribute

This feature introduces a new lock attribute in the project manifest. This attribute is populated by the Package Manager to render the package configuration resolution deterministic.

Supported URL schemes/protocols

The Package Manager supports the same URL schemes as Git does. In fact, the URL is passed to the Git executable (minus the revision part). If Git ever adds support for more protocol/schemes, they will automatically be supported by the Package Manager.

List of supported schemes

• git:
• ssh:
• https:
• http:
• _file: (_only URL-like path are supported for now. i.e. file://absolute/path/repo.git. System path will be supported later. This could allow using relative path for example file:…/…/repo.git)

We also support SCP style pattern:

git@github.com:user/repo.git

Revisions

You can select a specific revision to be cloned by the Package Manager. The revision is prefixed by the # character. The revision should be added after .git extension:

https://repository/path.git#revision

The revision can be any commit-ish - an expression that eventually resolves to a commit hash. The most common types of commit-ish are commit hashes, tags, and branches.

Limitations and good to know:

• The Package Manager UI was not updated to support this new package source (ETA 2019.1). I haven’t seen any more issue though.

• Big repositories can take a long time to clone. We don’t cache the repository so if you change revision, the repository will be cloned again.

• The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

• Since the Package Manager uses the Git executable installed on your machine. Any global or system configuration or environment variable will be picked up. These settings may affect the behaviour of the system.

• Some package managers offer a shorthand version of their supported URL schemes. For example, npm support short URLs like these: /, gitlab:/. We don’t support these syntaxes for the moment.

Hope this will help you test the feature.

Regards,

Pascal

[rz_0lento]

@okcompute_unity , thanks, that was very helpful. Any chance to get similar brief intro for scoped registries (private registries)?

[okcompute_unity]

Sure! Just start a thread for it

[falkj17]

@okcompute_unity thanks for that information! That is really useful!!
Is there any chance that the package manager will support finding packages in subdirectories of a git repository? Let’s say I have one repository that contains multiple packages in various subfolders of that repo. I want to target a specific one, would that be possible?

[okcompute_unity]

@falkj17 Yes, this is something on our roadmap. But we suggest putting your package at the root of the repository if possible. This will be easier for your users to find the package. For instance, users will have to know and specify the name of your package if it is deep in the repo. While if the package is at the root, only the URL will be required, the name will be optional.

[rz_0lento]

@okcompute_unity

Sure! Just start a thread for it

https://discussions.unity.com/t/719331

[falkj17]

Great, thanks! Any idea on how long it will take to add that functionality?

I completely agree that for visibility it would be better to have it in the root, but we would like to group them by different themes and have all relevant packages in one repo, if possible. Having every single package in a different repo also makes it more difficult to develop multiple packages in parallel

[rz_0lento]

IMO for development alone it’s better to keep the repos separated, but I could see use for being able to keep different packages in single repo as well, like if you keep the git as custom repository that just holds various open source assets as packages (I already have two separate repos for this purpose atm).

Also if you have like say, custom SRP fork right now, the repo contains many packages already and it would be handy if people could just ref to them directly.

Right now you can workaround this by just putting different packages into different branches/releases but it’s kinda hacky IMO.

[dzamani]

Hi, quick question about this:

@okcompute_unity

Hi @rz_0lento ,

Limitations and good to know:

• The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

If there isn’t any revision specified, is the package manager able to detect new version of the git package ?

In other dependency manager similar to this, you can for example specify an exact version match and it will go fetch that one that’s what specifying the revision looks like at the moment. But is there a way with git repos (I know I can do it with the scoped registries) to fetch and check if there is an update of the repo (based on the package.json) ?

Thanks again for this feature!

[rz_0lento]

@dzamani

If there isn’t any revision specified, is the package manager able to detect new version of the git package ?

if you don’t specify the revision, I think it will just use the default branch configured for the git repo, and to update it later on refer to:

@okcompute_unity

• The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

Package specific lock attribute is generated at the end of manifest once the package manager processes the git clone/update, it basically stores the hash from git repo and revision. If you clear it, it should update to latest.

[dzamani]

@rz_0lento

if you don’t specify the revision, I think it will just use the default branch configured for the git repo, and to update it later on refer to:

package specific lock attribute is generated at the end of manifest once the package manager processes the git clone/update, it basically stores the hash from git repo and revision, if you clear it, it should update to latest.

That’s what I was asking, dependency managers like Cocoapods will check if it’s the latest commit if you did not specify a revision.

So what I’m actually asking is : Is there any plan to handle this usecase where the package manager will check if he has the latest commit or not on a specific branch?

[M_R]

@okcompute_unity

We also support SCP style pattern:

git@github.com:user/repo.git

I tried this and it didn’t work.
I managed to make it work by specifying the url with the ssh scheme:
ssh://git@bitbucket.org/myorg/my-repo.git#1.0.0
git: scheme didn’t work. also note the slash “/” between host and org instead of colon “:”
http(s) scheme is not working as it complains about missing credentials (and I don’t want to specify my password in the url (that will get committed)) (note: my repos are all private)

[Rotary-Heart]

Is there anyway to add a dependency between git packages? Current dependency seems to only accept name and version:

    "dependencies": {
        "packagename": "1.1.1-preview"
    }

If a git link is added it will throw an error since it doesn’t recognize it as a version number:

    "dependencies": {
        "packagename": "https://gitprovider.com/MyPackage.git"
    }

Other than this, pretty amazing how easily is to setup the git links, great job!

[rz_0lento]

I renamed the thread for clarity as this has gone well past the syntax of the manifest entry

[quote=“Rotary-Heart, post:15, topic: 719263, username:Rotary-Heart”]
Is there anyway to add a dependency between git packages?
[/quote]I’d guess it’s the same deal as with direct folder refs atm (no dependency support). I do admit it would be handy but how should that be setup so it would work both locally and from git repos?

[Rotary-Heart]

@rz_0lento

I renamed the thread for clarity as this has gone well past the syntax of the manifest entry

I’d guess it’s the same deal as with direct folder refs atm (no dependency support).

I see, I missed that dependencies didn’t work with local folders either.

@rz_0lento

I do admit it would be handy but how should that be setup so it would work both locally and from git repos?

I would guess that the same way you setup them on your project manifest. All it really needs is to point to the location where the other package.json is located and process it.

[andybak]

@okcompute_unity

@falkj17 Yes, this is something on our roadmap. But we suggest putting your package at the root of the repository if possible.

I think you’re missing a major use case for this. There is a LOT of great stuff already on Github and it usually ends up cluttering up Asset folders because there’s no other way to bring it in to a project.

If you supported packages in subdirectories we could:

1. Fork the repo we want to use
2. Add a package.json to the correct directory.

Most Unity repos are entire projects. Forking and adding a package.json to the root dir won’t work in most cases. Forking and rearranging the directory structure makes it impossible to track upstream changes.

I’ve experimented with submodules, sparse checkouts and similar to get this working but they are extremely fragile.

[rz_0lento]

[quote=“andybak, post:18, topic: 719263, username:andybak”]
it usually ends up cluttering up Asset folders because there’s no other way to bring it in to a project.
[/quote]If you share whole project IMO best place for custom packages is just to place them into Packages folder for two reasons: you don’t have to modify manifest at all, Unity picks them from there as packages automatically and it also keeps your Assets folder clean as they are not there bloating it.

I’d love to be able to point into that git repos Packages/ path directly from the manifest as this would let people share whole projects and also at the same time let others pick the package only from the project.

[andybak]

@rz_0lento

If you share whole project IMO best place for custom packages is just to place them into Packages folder for two reasons: you don’t have to modify manifest at all, Unity picks them from there as packages automatically and it also keeps your Assets folder clean as they are not there bloating it.

I’d love to be able to point into that git repos Packages/ path directly from the manifest as this would let people share whole projects and also at the same time let others pick the package only from the project.

Unless I’m misreading you - you’re making a slightly orthogonal point (which I agree with)?

Just checking I haven’t misunderstood.