Using GitHub Packages Registry with Unity Package Manager

Hi,

Thanks to forum user @JakHussain , there is now a known method to use GitHub Packages Registry with Unity Package Manager. See the original post where @JakHussain shared his clever finding.

WARNING: Using GitHub Packages registries with Unity Package Manager comes with a couple of limitations (Goto the end of the post for the list). Please be aware that we are posting this only to share community knowledge. Unity doesn’t officially support GitHub Packages Registry. Use it at your own risk.

There are a couple of steps needed to make it work.

1 - You need to configure the publishConfig setting as described in GitHub Packages Registry documentation in a special way. You need to suffix the pakcage scope:

  "publishConfig": { "registry": "https://npm.pkg.github.com/@USER" },

Where USER is the Github organization or user (i.e.the scope). Basically, we are cheating on how NPM works and inject the scope in the registry URL instead of prefixing the package name. This is one half of the trick. Unity Package Manager doesn’t accept the @ and / characters in package names. By moving the scope reference in the registry URL we don’t need these characters in the package name anymore!

Add the publisConfig configuration in your package manifest (package.json)

2 - Set the name of your package without an NPM scope

This is the other part of the magic. As stated in step #1, make sure you don’t prefix you package name with an NPM scope ( @ )

Example:

  "name": "com.my-company.my-package",

Not:

  "name": "@USER/com.my-company.my-package",

WARNING: If you have previously published a package with an NPM scope in the name, it won’t work. Your package should not have an MPM scope for all existing versions. It won’t work otherwise.

3 - Publish your package

For that step, you can refer to GitHub documentation.

4- Configure the scope registry in your Unity project manifest

  "scopedRegistries": [
    {
      "name": "Company Packages",
      "url": "https://npm.pkg.github.com/@USER",
      "scopes": [
        "com.my-company"
      ]
    }
  ],

The URL must match the one configured in your publishConfig above in step #1.

5 - Setup Npm Auth configuration

WARNING: GitHub Packages registry always requires authentication. For private and public packages.

For this one, I’ll refer you to another post I’ve made explaining how .

6 - Add your package in your project manifest

GitHub Package Registry packages won’t show up in the Package Manager UI All section. You have to install the package manually in your project manifest.

Known limitations

  • The main limitation is discoverability. Even if your Github Packages registry-based scoped registry is correctly configured, packages won’t show up in the Package Manager UI’s All section. GitHub doesn’t support any of the NPM search API endpoints (/-/all or /-/v1/search). You will have to resort to GitHub UI to search packages and manually add them to your project manifest.
  • Even if the GitHub Registry package is public, your users will need to provide authentication.

As you can see, using GitHub Packages Registry with Unity Package Manager is not trivial. But, if you really need it, you can make it work!

Hope this post will be useful to some of you. Again, thank you to @JakHussain for discovering how it can be done!

Regards,

Pascal and the Unity Package Manager team.

12 Likes

This is sad. We should have option to use

"publishConfig": { "registry": "https://npm.pkg.github.com" },

// with

"name": "USER/com.my-company.my-package", // without @

Instead. This would be easier to orchestrate cross domain dependency. I may publish project that utilize project of other person or other organization easily. It would critically expand the usage of package manager over everything in one go with just one line of registry

1 Like

@Thaina that’s down to how githubs package registry works rather than how unitys package manager works unfortunately. So far my experience with GitHub Packages has been pretty underwhelming since they really don’t make it easy for you to use. Fingers crossed for the future though!

3 Likes

Why this? This excludes GitLab for use since GitLab requires scopes to be present and rejects packages without scopes.
Seems that your suggested cheat does not work with Gitlab. And cheating should never be necessary (strictly against cheating, makes life harder for all of us).

‘Packages must be scoped in the root namespace of the project.’
from https://docs.gitlab.com/ee/user/packages/npm_registry/#package-naming-convention

1 Like

I would chalk this up to githubs package registry forcing our hand in cheating in this way. As far as I can tell, unitys package manager isn’t the bottle neck in how packages are read and pulled in.

Thanks for this guide. I tested it and it seems to work. We have the need to share with outside resources and this will help, although a pain to setup.

We are using some custom tooling to manage/publish NPM packages, and this could be added to it.

Is there a future where you think packages with the naming scheme @anon_3270756 /package-name will be supported? It seems that this is enforced by multiple NPM repository hosts. An option could be to ignore @anon_3270756 / in the name except when pulling from the NPM repo, so the full package name will become “@vendor/tld.vendor.package-name”

2 Likes

Hi @JesperSmith ,

As you pointed out, the Gitthub Packages Registry is not easy to use with the Package Manager. Since it lacks discoverability API, we will not put more effort to try to be compatible with this registry. Maybe you can reach out to GitHub and ask them to support UPM registries format (including documentation, adapted API, etc.)?

Regards,

Pascal

4 Likes

Fair enough. I’ll see if we can make some extra tooling on our side to make it work well enough for our purposes.

Unity would not need to support Github natively. We would even be capable of developing an extension on our own to discover packages on any registry which makes this somehow possible via an API. I don’t belive GitHub will go implementing UPM registries format.

Instead we are asking to support the NPM Scoping syntax which we will not be able to support via an extension on our own (except of hacking around it which would end up in writing our own package manager)
This is because NPM registries like the one of GitLab (not GitHub…) requires a scope for packages and this is why unity editor can not import those packages from GitLab.

(Maybe there is a better thread for this or there should be one)

2 Likes

I think the issue been discussed a few times. Supporting /-/all/ discovery endpoint is a better-to-have thing. What really blocks this is the scopeRegistry syntax rejects “@” letter. Unity decides to use dot notation as the scope, but other NPM based registry defaults to @ syntax. It didn’t sound like a tech limitation but a design decision that whether @ conflicts with dot notation. Maybe there’re other reasons behind, but for me, below are all valid scopes.

  • com.company

  • @

  • @ /com.company

Sure, there’re other registries that won’t force you to use @ as scope for the private package. But it’s kinda convention here for NPM, GitHub and GitLab. They’re all big players on the market.

6 Likes

Using 2019.3.7f1 we have been unable to replicate this behavior. UPM fails to pull in the manually specified package due to an inability to find it. Using npm install with the same token pointed towards the same URL works perfectly.

This forum thread is only a little less than 2 months old, so it seems a bit strange that the behavior would already have diverged.

This workaround seems to have been broken by a recent change by Github:

They no longer allow publishing of packages without the scope in the name.

@okcompute_unity - do you know of any way that we can work around this and continue to use the Github Package Registry with Unity 2019.4?

Alternatively, can you provide any suggestions for an alternative private package manager, that does not require us to host our own package manager (i.e. we are not really interested in self hosting verdaccio)?

Best regards,
Morten

1 Like

Hi @TactileGames ,

I’m sorry you were affected by this change from Github. Some discussions are happening right now between Unity and Github. I cannot promise anything but fingers crossed!

In the meantime, a self-hosting solution that I can recommend is JFrog’s Artifactory (https://jfrog.com/artifactory/). Easy setup and they offer a free tier that can be interesting for a small team/studio.

Regards,

Pascal

2 Likes

I was told this morning by Github that they reverted the change they made:
https://github.community/t/unable-to-publish-packages-with-different-name-to-publish-url-as-of-07-10-2020/136643/11?u=jcansdale

So the method described in this post should work again. Note that I was advised this is temporary but it should work for a while. The next major version of their system may not support this anymore.

Reminder: Unity doesn’t officially support the Github Packages registry. Github did not implement all the npm API UPM relies on for the scoped registries feature.

Regards,

Pascal

@okcompute_unity
Excellent that it works again.

What is the reason that the Unity Package Manager doesn’t allow package names beginning with an @ scope?

@okcompute_unity Thank you for the guide ! It works, I succeed to publish my package on GitHub and download it on a project with the scoped registry.

There is just one issue on the Package Manager. As you said, GitHub doesn’t support for searching, but the Package Manager will try anyway and it “crash” it a bit.

I’m on the “All packages” section, but once it failed to search on GitHub, it won’t load any others packages … It could be nice to just display a warning or fail silently.

For information, i’m on 2019.4.5f1

I’ve gotten as far as steps 4/5/6 - not sure which bit is failing. Package is hosted on a private GitHub organization repo.

Unity gives me.

I’ve got the .upmconfig.toml setup in Users/MyLogin/ with

The project manifest.json is like so:

Has anyone tried to have package dependencies and if you need to do the following with the scope @ /com.test.dependency ?

Will the Unity Package Manager support npm’s scope notation in future versions? We use GitLab, and it can host npm packages, but only when using exactly that (" @anon_3270756 /package.name") naming convention. So after getting my package published on our GitLab server, I found out that all my work was for nothing because the UPM is not fully npm compatible.
Should have read this thread first I guess. But seeing that GitHub also tried moving into the scoped naming requirement, are you going to fix this?

In our view, this is not something that needs to be fixed, but a new feature request that we don’t plan to implement at this time. We don’t plan to support npm’s scope notation in the package names, a package with the name @foo/bar is not valid.