Add Go package registry (#24687)

Fixes #7608

This PR adds a Go package registry usable with the Go proxy protocol.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/328feb5c-3df2-4f9d-8eae-fe3126d14c37)
This commit is contained in:
KN4CK3R 2023-05-14 17:38:40 +02:00 committed by GitHub
parent 53a00017bb
commit 5968c63a11
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
23 changed files with 751 additions and 10 deletions

View File

@ -2463,6 +2463,8 @@ ROUTER = console
;LIMIT_SIZE_DEBIAN = -1 ;LIMIT_SIZE_DEBIAN = -1
;; Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) ;; Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_GENERIC = -1 ;LIMIT_SIZE_GENERIC = -1
;; Maximum size of a Go upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_GO = -1
;; Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) ;; Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_HELM = -1 ;LIMIT_SIZE_HELM = -1
;; Maximum size of a Maven upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) ;; Maximum size of a Maven upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)

View File

@ -1223,6 +1223,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
- `LIMIT_SIZE_CONTAINER`: **-1**: Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_CONTAINER`: **-1**: Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_DEBIAN`: **-1**: Maximum size of a Debian upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_DEBIAN`: **-1**: Maximum size of a Debian upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_GENERIC`: **-1**: Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_GENERIC`: **-1**: Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_GO`: **-1**: Maximum size of a Go upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_HELM`: **-1**: Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_HELM`: **-1**: Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_MAVEN`: **-1**: Maximum size of a Maven upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_MAVEN`: **-1**: Maximum size of a Maven upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_NPM`: **-1**: Maximum size of a npm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`) - `LIMIT_SIZE_NPM`: **-1**: Maximum size of a npm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)

View File

@ -83,7 +83,7 @@ curl --user your_username:your_password_or_token \
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
You cannot publish a file with the same name twice to a package. You must delete the existing package version first. You cannot publish a file with the same name twice to a package. You must delete the existing package version first.
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |
@ -115,7 +115,7 @@ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/debian/pools/bionic/main/test-package/1.0.0/amd64 https://gitea.example.com/api/packages/testuser/debian/pools/bionic/main/test-package/1.0.0/amd64
``` ```
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |

View File

@ -51,7 +51,7 @@ curl --user your_username:your_password_or_token \
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |
@ -83,7 +83,7 @@ curl --user your_username:your_token_or_password \
https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0/file.bin https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0/file.bin
``` ```
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |
@ -111,7 +111,7 @@ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0 https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0
``` ```
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |
@ -140,7 +140,7 @@ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0/file.bin https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0/file.bin
``` ```
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |

View File

@ -0,0 +1,77 @@
---
date: "2023-05-10T00:00:00+00:00"
title: "Go Packages Repository"
slug: "go"
weight: 45
draft: false
toc: false
menu:
sidebar:
parent: "packages"
name: "Go"
weight: 45
identifier: "go"
---
# Go Packages Repository
Publish Go packages for your user or organization.
**Table of Contents**
{{< toc >}}
## Publish a package
To publish a Go package perform a HTTP `PUT` operation with the package content in the request body.
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first.
The package must follow the [documented structure](https://go.dev/ref/mod#zip-files).
```
PUT https://gitea.example.com/api/packages/{owner}/go/upload
```
| Parameter | Description |
| --------- | ----------- |
| `owner` | The owner of the package. |
To authenticate to the package registry, you need to provide [custom HTTP headers or use HTTP Basic authentication]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}):
```shell
curl --user your_username:your_password_or_token \
--upload-file path/to/file.zip \
https://gitea.example.com/api/packages/testuser/go/upload
```
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `201 Created` | The package has been published. |
| `400 Bad Request` | The package is invalid. |
| `409 Conflict` | A package with the same name exist already. |
## Install a package
To install a Go package instruct Go to use the package registry as proxy:
```shell
# use latest version
GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name}
# or
GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name}@latest
# use specific version
GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name}@{package_version}
```
| Parameter | Description |
| ----------------- | ----------- |
| `owner` | The owner of the package. |
| `package_name` | The package name. |
| `package_version` | The package version. |
If the owner of the packages is private you need to [provide credentials](https://go.dev/ref/mod#private-module-proxy-auth).
More information about the `GOPROXY` environment variable and how to protect against data leaks can be found in [the documentation](https://go.dev/ref/mod#private-modules).

View File

@ -69,7 +69,7 @@ curl --user your_username:your_password_or_token \
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
You cannot publish a file with the same name twice to a package. You must delete the existing package version first. You cannot publish a file with the same name twice to a package. You must delete the existing package version first.
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |
@ -99,7 +99,7 @@ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64 https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64
``` ```
The server reponds with the following HTTP Status codes. The server responds with the following HTTP Status codes.
| HTTP Status Code | Meaning | | HTTP Status Code | Meaning |
| ----------------- | ------- | | ----------------- | ------- |

View File

@ -155,6 +155,8 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc
metadata = &debian.Metadata{} metadata = &debian.Metadata{}
case TypeGeneric: case TypeGeneric:
// generic packages have no metadata // generic packages have no metadata
case TypeGo:
// go packages have no metadata
case TypeHelm: case TypeHelm:
metadata = &helm.Metadata{} metadata = &helm.Metadata{}
case TypeNuGet: case TypeNuGet:

View File

@ -39,6 +39,7 @@ const (
TypeContainer Type = "container" TypeContainer Type = "container"
TypeDebian Type = "debian" TypeDebian Type = "debian"
TypeGeneric Type = "generic" TypeGeneric Type = "generic"
TypeGo Type = "go"
TypeHelm Type = "helm" TypeHelm Type = "helm"
TypeMaven Type = "maven" TypeMaven Type = "maven"
TypeNpm Type = "npm" TypeNpm Type = "npm"
@ -61,6 +62,7 @@ var TypeList = []Type{
TypeContainer, TypeContainer,
TypeDebian, TypeDebian,
TypeGeneric, TypeGeneric,
TypeGo,
TypeHelm, TypeHelm,
TypeMaven, TypeMaven,
TypeNpm, TypeNpm,
@ -94,6 +96,8 @@ func (pt Type) Name() string {
return "Debian" return "Debian"
case TypeGeneric: case TypeGeneric:
return "Generic" return "Generic"
case TypeGo:
return "Go"
case TypeHelm: case TypeHelm:
return "Helm" return "Helm"
case TypeMaven: case TypeMaven:
@ -139,6 +143,8 @@ func (pt Type) SVGName() string {
return "gitea-debian" return "gitea-debian"
case TypeGeneric: case TypeGeneric:
return "octicon-package" return "octicon-package"
case TypeGo:
return "gitea-go"
case TypeHelm: case TypeHelm:
return "gitea-helm" return "gitea-helm"
case TypeMaven: case TypeMaven:

View File

@ -0,0 +1,94 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package goproxy
import (
"archive/zip"
"fmt"
"io"
"path"
"strings"
"code.gitea.io/gitea/modules/util"
)
const (
PropertyGoMod = "go.mod"
maxGoModFileSize = 16 * 1024 * 1024 // https://go.dev/ref/mod#zip-path-size-constraints
)
var (
ErrInvalidStructure = util.NewInvalidArgumentErrorf("package has invalid structure")
ErrGoModFileTooLarge = util.NewInvalidArgumentErrorf("go.mod file is too large")
)
type Package struct {
Name string
Version string
GoMod string
}
// ParsePackage parses the Go package file
// https://go.dev/ref/mod#zip-files
func ParsePackage(r io.ReaderAt, size int64) (*Package, error) {
archive, err := zip.NewReader(r, size)
if err != nil {
return nil, err
}
var p *Package
for _, file := range archive.File {
nameAndVersion := path.Dir(file.Name)
parts := strings.SplitN(nameAndVersion, "@", 2)
if len(parts) != 2 {
continue
}
versionParts := strings.SplitN(parts[1], "/", 2)
if p == nil {
p = &Package{
Name: strings.TrimSuffix(nameAndVersion, "@"+parts[1]),
Version: versionParts[0],
}
}
if len(versionParts) > 1 {
// files are expected in the "root" folder
continue
}
if path.Base(file.Name) == "go.mod" {
if file.UncompressedSize64 > maxGoModFileSize {
return nil, ErrGoModFileTooLarge
}
f, err := archive.Open(file.Name)
if err != nil {
return nil, err
}
defer f.Close()
bytes, err := io.ReadAll(&io.LimitedReader{R: f, N: maxGoModFileSize})
if err != nil {
return nil, err
}
p.GoMod = string(bytes)
return p, nil
}
}
if p == nil {
return nil, ErrInvalidStructure
}
p.GoMod = fmt.Sprintf("module %s", p.Name)
return p, nil
}

View File

@ -0,0 +1,75 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package goproxy
import (
"archive/zip"
"bytes"
"testing"
"github.com/stretchr/testify/assert"
)
const (
packageName = "gitea.com/go-gitea/gitea"
packageVersion = "v0.0.1"
)
func TestParsePackage(t *testing.T) {
createArchive := func(files map[string][]byte) *bytes.Reader {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for name, content := range files {
w, _ := zw.Create(name)
w.Write(content)
}
zw.Close()
return bytes.NewReader(buf.Bytes())
}
t.Run("EmptyPackage", func(t *testing.T) {
data := createArchive(nil)
p, err := ParsePackage(data, int64(data.Len()))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidStructure)
})
t.Run("InvalidNameOrVersionStructure", func(t *testing.T) {
data := createArchive(map[string][]byte{
packageName + "/" + packageVersion + "/go.mod": {},
})
p, err := ParsePackage(data, int64(data.Len()))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidStructure)
})
t.Run("GoModFileInWrongDirectory", func(t *testing.T) {
data := createArchive(map[string][]byte{
packageName + "@" + packageVersion + "/subdir/go.mod": {},
})
p, err := ParsePackage(data, int64(data.Len()))
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, packageName, p.Name)
assert.Equal(t, packageVersion, p.Version)
assert.Equal(t, "module gitea.com/go-gitea/gitea", p.GoMod)
})
t.Run("Valid", func(t *testing.T) {
data := createArchive(map[string][]byte{
packageName + "@" + packageVersion + "/subdir/go.mod": []byte("invalid"),
packageName + "@" + packageVersion + "/go.mod": []byte("valid"),
})
p, err := ParsePackage(data, int64(data.Len()))
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, packageName, p.Name)
assert.Equal(t, packageVersion, p.Version)
assert.Equal(t, "valid", p.GoMod)
})
}

View File

@ -33,6 +33,7 @@ var (
LimitSizeContainer int64 LimitSizeContainer int64
LimitSizeDebian int64 LimitSizeDebian int64
LimitSizeGeneric int64 LimitSizeGeneric int64
LimitSizeGo int64
LimitSizeHelm int64 LimitSizeHelm int64
LimitSizeMaven int64 LimitSizeMaven int64
LimitSizeNpm int64 LimitSizeNpm int64
@ -79,6 +80,7 @@ func loadPackagesFrom(rootCfg ConfigProvider) {
Packages.LimitSizeContainer = mustBytes(sec, "LIMIT_SIZE_CONTAINER") Packages.LimitSizeContainer = mustBytes(sec, "LIMIT_SIZE_CONTAINER")
Packages.LimitSizeDebian = mustBytes(sec, "LIMIT_SIZE_DEBIAN") Packages.LimitSizeDebian = mustBytes(sec, "LIMIT_SIZE_DEBIAN")
Packages.LimitSizeGeneric = mustBytes(sec, "LIMIT_SIZE_GENERIC") Packages.LimitSizeGeneric = mustBytes(sec, "LIMIT_SIZE_GENERIC")
Packages.LimitSizeGo = mustBytes(sec, "LIMIT_SIZE_GO")
Packages.LimitSizeHelm = mustBytes(sec, "LIMIT_SIZE_HELM") Packages.LimitSizeHelm = mustBytes(sec, "LIMIT_SIZE_HELM")
Packages.LimitSizeMaven = mustBytes(sec, "LIMIT_SIZE_MAVEN") Packages.LimitSizeMaven = mustBytes(sec, "LIMIT_SIZE_MAVEN")
Packages.LimitSizeNpm = mustBytes(sec, "LIMIT_SIZE_NPM") Packages.LimitSizeNpm = mustBytes(sec, "LIMIT_SIZE_NPM")

View File

@ -3263,6 +3263,8 @@ debian.repository.components = Components
debian.repository.architectures = Architectures debian.repository.architectures = Architectures
generic.download = Download package from the command line: generic.download = Download package from the command line:
generic.documentation = For more information on the generic registry, see <a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>. generic.documentation = For more information on the generic registry, see <a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>.
go.install = Install the package from the command line:
go.documentation = For more information on the Go registry, see <a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>.
helm.registry = Setup this registry from the command line: helm.registry = Setup this registry from the command line:
helm.install = To install the package, run the following command: helm.install = To install the package, run the following command:
helm.documentation = For more information on the Helm registry, see <a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>. helm.documentation = For more information on the Helm registry, see <a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>.

1
public/img/svg/gitea-go.svg generated Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="24.7 77.83 205.42 76.8" class="svg gitea-go" width="16" height="16" aria-hidden="true"><g style="fill:#00acd7"><path d="M40.2 101.1c-.4 0-.5-.2-.3-.5l2.1-2.7c.2-.3.7-.5 1.1-.5h35.7c.4 0 .5.3.3.6l-1.7 2.6c-.2.3-.7.6-1 .6l-36.2-.1zM25.1 110.3c-.4 0-.5-.2-.3-.5l2.1-2.7c.2-.3.7-.5 1.1-.5h45.6c.4 0 .6.3.5.6l-.8 2.4c-.1.4-.5.6-.9.6l-47.3.1zM49.3 119.5c-.4 0-.5-.3-.3-.6l1.4-2.5c.2-.3.6-.6 1-.6h20c.4 0 .6.3.6.7l-.2 2.4c0 .4-.4.7-.7.7l-21.8-.1zM153.1 99.3c-6.3 1.6-10.6 2.8-16.8 4.4-1.5.4-1.6.5-2.9-1-1.5-1.7-2.6-2.8-4.7-3.8-6.3-3.1-12.4-2.2-18.1 1.5-6.8 4.4-10.3 10.9-10.2 19 .1 8 5.6 14.6 13.5 15.7 6.8.9 12.5-1.5 17-6.6.9-1.1 1.7-2.3 2.7-3.7h-19.3c-2.1 0-2.6-1.3-1.9-3 1.3-3.1 3.7-8.3 5.1-10.9.3-.6 1-1.6 2.5-1.6h36.4c-.2 2.7-.2 5.4-.6 8.1-1.1 7.2-3.8 13.8-8.2 19.6-7.2 9.5-16.6 15.4-28.5 17-9.8 1.3-18.9-.6-26.9-6.6-7.4-5.6-11.6-13-12.7-22.2-1.3-10.9 1.9-20.7 8.5-29.3 7.1-9.3 16.5-15.2 28-17.3 9.4-1.7 18.4-.6 26.5 4.9 5.3 3.5 9.1 8.3 11.6 14.1.6.9.2 1.4-1 1.7z"/><path d="M186.2 154.6c-9.1-.2-17.4-2.8-24.4-8.8-5.9-5.1-9.6-11.6-10.8-19.3-1.8-11.3 1.3-21.3 8.1-30.2 7.3-9.6 16.1-14.6 28-16.7 10.2-1.8 19.8-.8 28.5 5.1 7.9 5.4 12.8 12.7 14.1 22.3 1.7 13.5-2.2 24.5-11.5 33.9-6.6 6.7-14.7 10.9-24 12.8-2.7.5-5.4.6-8 .9zm23.8-40.4c-.1-1.3-.1-2.3-.3-3.3-1.8-9.9-10.9-15.5-20.4-13.3-9.3 2.1-15.3 8-17.5 17.4-1.8 7.8 2 15.7 9.2 18.9 5.5 2.4 11 2.1 16.3-.6 7.9-4.1 12.2-10.5 12.7-19.1z"/></g></svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

@ -24,6 +24,7 @@ import (
"code.gitea.io/gitea/routers/api/packages/container" "code.gitea.io/gitea/routers/api/packages/container"
"code.gitea.io/gitea/routers/api/packages/debian" "code.gitea.io/gitea/routers/api/packages/debian"
"code.gitea.io/gitea/routers/api/packages/generic" "code.gitea.io/gitea/routers/api/packages/generic"
"code.gitea.io/gitea/routers/api/packages/goproxy"
"code.gitea.io/gitea/routers/api/packages/helm" "code.gitea.io/gitea/routers/api/packages/helm"
"code.gitea.io/gitea/routers/api/packages/maven" "code.gitea.io/gitea/routers/api/packages/maven"
"code.gitea.io/gitea/routers/api/packages/npm" "code.gitea.io/gitea/routers/api/packages/npm"
@ -312,6 +313,64 @@ func CommonRoutes(ctx gocontext.Context) *web.Route {
}, reqPackageAccess(perm.AccessModeWrite)) }, reqPackageAccess(perm.AccessModeWrite))
}) })
}, reqPackageAccess(perm.AccessModeRead)) }, reqPackageAccess(perm.AccessModeRead))
r.Group("/go", func() {
r.Put("/upload", reqPackageAccess(perm.AccessModeWrite), goproxy.UploadPackage)
r.Get("/sumdb/sum.golang.org/supported", func(ctx *context.Context) {
ctx.Status(http.StatusNotFound)
})
// Manual mapping of routes because the package name contains slashes which chi does not support
// https://go.dev/ref/mod#goproxy-protocol
r.Get("/*", func(ctx *context.Context) {
path := ctx.Params("*")
if strings.HasSuffix(path, "/@latest") {
ctx.SetParams("name", path[:len(path)-len("/@latest")])
ctx.SetParams("version", "latest")
goproxy.PackageVersionMetadata(ctx)
return
}
parts := strings.SplitN(path, "/@v/", 2)
if len(parts) != 2 {
ctx.Status(http.StatusNotFound)
return
}
ctx.SetParams("name", parts[0])
// <package/name>/@v/list
if parts[1] == "list" {
goproxy.EnumeratePackageVersions(ctx)
return
}
// <package/name>/@v/<version>.zip
if strings.HasSuffix(parts[1], ".zip") {
ctx.SetParams("version", parts[1][:len(parts[1])-len(".zip")])
goproxy.DownloadPackageFile(ctx)
return
}
// <package/name>/@v/<version>.info
if strings.HasSuffix(parts[1], ".info") {
ctx.SetParams("version", parts[1][:len(parts[1])-len(".info")])
goproxy.PackageVersionMetadata(ctx)
return
}
// <package/name>/@v/<version>.mod
if strings.HasSuffix(parts[1], ".mod") {
ctx.SetParams("version", parts[1][:len(parts[1])-len(".mod")])
goproxy.PackageVersionGoModContent(ctx)
return
}
ctx.Status(http.StatusNotFound)
})
}, reqPackageAccess(perm.AccessModeRead))
r.Group("/generic", func() { r.Group("/generic", func() {
r.Group("/{packagename}/{packageversion}", func() { r.Group("/{packagename}/{packageversion}", func() {
r.Delete("", reqPackageAccess(perm.AccessModeWrite), generic.DeletePackage) r.Delete("", reqPackageAccess(perm.AccessModeWrite), generic.DeletePackage)

View File

@ -0,0 +1,226 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package goproxy
import (
"errors"
"fmt"
"io"
"net/http"
"sort"
"time"
packages_model "code.gitea.io/gitea/models/packages"
"code.gitea.io/gitea/modules/context"
packages_module "code.gitea.io/gitea/modules/packages"
goproxy_module "code.gitea.io/gitea/modules/packages/goproxy"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/routers/api/packages/helper"
packages_service "code.gitea.io/gitea/services/packages"
)
func apiError(ctx *context.Context, status int, obj interface{}) {
helper.LogAndProcessError(ctx, status, obj, func(message string) {
ctx.PlainText(status, message)
})
}
func EnumeratePackageVersions(ctx *context.Context) {
pvs, err := packages_model.GetVersionsByPackageName(ctx, ctx.Package.Owner.ID, packages_model.TypeGo, ctx.Params("name"))
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
if len(pvs) == 0 {
apiError(ctx, http.StatusNotFound, err)
return
}
sort.Slice(pvs, func(i, j int) bool {
return pvs[i].CreatedUnix < pvs[j].CreatedUnix
})
ctx.Resp.Header().Set("Content-Type", "text/plain;charset=utf-8")
for _, pv := range pvs {
fmt.Fprintln(ctx.Resp, pv.Version)
}
}
func PackageVersionMetadata(ctx *context.Context) {
pv, err := resolvePackage(ctx, ctx.Package.Owner.ID, ctx.Params("name"), ctx.Params("version"))
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
ctx.JSON(http.StatusOK, struct {
Version string `json:"Version"`
Time time.Time `json:"Time"`
}{
Version: pv.Version,
Time: pv.CreatedUnix.AsLocalTime(),
})
}
func PackageVersionGoModContent(ctx *context.Context) {
pv, err := resolvePackage(ctx, ctx.Package.Owner.ID, ctx.Params("name"), ctx.Params("version"))
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
pps, err := packages_model.GetPropertiesByName(ctx, packages_model.PropertyTypeVersion, pv.ID, goproxy_module.PropertyGoMod)
if err != nil || len(pps) != 1 {
apiError(ctx, http.StatusInternalServerError, err)
return
}
ctx.PlainText(http.StatusOK, pps[0].Value)
}
func DownloadPackageFile(ctx *context.Context) {
pv, err := resolvePackage(ctx, ctx.Package.Owner.ID, ctx.Params("name"), ctx.Params("version"))
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
pfs, err := packages_model.GetFilesByVersionID(ctx, pv.ID)
if err != nil || len(pfs) != 1 {
apiError(ctx, http.StatusInternalServerError, err)
return
}
s, _, err := packages_service.GetPackageFileStream(ctx, pfs[0])
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
defer s.Close()
ctx.ServeContent(s, &context.ServeHeaderOptions{
Filename: pfs[0].Name,
LastModified: pfs[0].CreatedUnix.AsLocalTime(),
})
}
func resolvePackage(ctx *context.Context, ownerID int64, name, version string) (*packages_model.PackageVersion, error) {
var pv *packages_model.PackageVersion
if version == "latest" {
pvs, _, err := packages_model.SearchLatestVersions(ctx, &packages_model.PackageSearchOptions{
OwnerID: ownerID,
Type: packages_model.TypeGo,
Name: packages_model.SearchValue{
Value: name,
ExactMatch: true,
},
IsInternal: util.OptionalBoolFalse,
Sort: packages_model.SortCreatedDesc,
})
if err != nil {
return nil, err
}
if len(pvs) != 1 {
return nil, packages_model.ErrPackageNotExist
}
pv = pvs[0]
} else {
var err error
pv, err = packages_model.GetVersionByNameAndVersion(ctx, ownerID, packages_model.TypeGo, name, version)
if err != nil {
return nil, err
}
}
return pv, nil
}
func UploadPackage(ctx *context.Context) {
upload, close, err := ctx.UploadStream()
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
if close {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
defer buf.Close()
pck, err := goproxy_module.ParsePackage(buf, buf.Size())
if err != nil {
if errors.Is(err, util.ErrInvalidArgument) {
apiError(ctx, http.StatusBadRequest, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
if _, err := buf.Seek(0, io.SeekStart); err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
_, _, err = packages_service.CreatePackageAndAddFile(
&packages_service.PackageCreationInfo{
PackageInfo: packages_service.PackageInfo{
Owner: ctx.Package.Owner,
PackageType: packages_model.TypeGo,
Name: pck.Name,
Version: pck.Version,
},
Creator: ctx.Doer,
VersionProperties: map[string]string{
goproxy_module.PropertyGoMod: pck.GoMod,
},
},
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%v.zip", pck.Version),
},
Creator: ctx.Doer,
Data: buf,
IsLead: true,
},
)
if err != nil {
switch err {
case packages_model.ErrDuplicatePackageVersion:
apiError(ctx, http.StatusConflict, err)
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
apiError(ctx, http.StatusForbidden, err)
default:
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
ctx.Status(http.StatusCreated)
}

View File

@ -40,7 +40,7 @@ func ListPackages(ctx *context.APIContext) {
// in: query // in: query
// description: package type filter // description: package type filter
// type: string // type: string
// enum: [alpine, cargo, chef, composer, conan, conda, container, debian, generic, helm, maven, npm, nuget, pub, pypi, rpm, rubygems, swift, vagrant] // enum: [alpine, cargo, chef, composer, conan, conda, container, debian, generic, go, helm, maven, npm, nuget, pub, pypi, rpm, rubygems, swift, vagrant]
// - name: q // - name: q
// in: query // in: query
// description: name filter // description: name filter

View File

@ -15,7 +15,7 @@ import (
type PackageCleanupRuleForm struct { type PackageCleanupRuleForm struct {
ID int64 ID int64
Enabled bool Enabled bool
Type string `binding:"Required;In(alpine,cargo,chef,composer,conan,conda,container,debian,generic,helm,maven,npm,nuget,pub,pypi,rpm,rubygems,swift,vagrant)"` Type string `binding:"Required;In(alpine,cargo,chef,composer,conan,conda,container,debian,generic,go,helm,maven,npm,nuget,pub,pypi,rpm,rubygems,swift,vagrant)"`
KeepCount int `binding:"In(0,1,5,10,25,50,100)"` KeepCount int `binding:"In(0,1,5,10,25,50,100)"`
KeepPattern string `binding:"RegexPattern"` KeepPattern string `binding:"RegexPattern"`
RemoveDays int `binding:"In(0,7,14,30,60,90,180)"` RemoveDays int `binding:"In(0,7,14,30,60,90,180)"`

View File

@ -369,6 +369,8 @@ func CheckSizeQuotaExceeded(ctx context.Context, doer, owner *user_model.User, p
typeSpecificSize = setting.Packages.LimitSizeDebian typeSpecificSize = setting.Packages.LimitSizeDebian
case packages_model.TypeGeneric: case packages_model.TypeGeneric:
typeSpecificSize = setting.Packages.LimitSizeGeneric typeSpecificSize = setting.Packages.LimitSizeGeneric
case packages_model.TypeGo:
typeSpecificSize = setting.Packages.LimitSizeGo
case packages_model.TypeHelm: case packages_model.TypeHelm:
typeSpecificSize = setting.Packages.LimitSizeHelm typeSpecificSize = setting.Packages.LimitSizeHelm
case packages_model.TypeMaven: case packages_model.TypeMaven:

View File

@ -0,0 +1,14 @@
{{if eq .PackageDescriptor.Package.Type "go"}}
<h4 class="ui top attached header">{{.locale.Tr "packages.installation"}}</h4>
<div class="ui attached segment">
<div class="ui form">
<div class="field">
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.go.install"}}</label>
<div class="markup"><pre class="code-block"><code>GOPROXY=<gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/go"></gitea-origin-url> go install {{$.PackageDescriptor.Package.Name}}@{{$.PackageDescriptor.Version.Version}}</code></pre></div>
</div>
<div class="field">
<label>{{.locale.Tr "packages.go.documentation" "https://docs.gitea.io/en-us/usage/packages/go" | Safe}}</label>
</div>
</div>
</div>
{{end}}

View File

@ -28,6 +28,7 @@
{{template "package/content/container" .}} {{template "package/content/container" .}}
{{template "package/content/debian" .}} {{template "package/content/debian" .}}
{{template "package/content/generic" .}} {{template "package/content/generic" .}}
{{template "package/content/go" .}}
{{template "package/content/helm" .}} {{template "package/content/helm" .}}
{{template "package/content/maven" .}} {{template "package/content/maven" .}}
{{template "package/content/npm" .}} {{template "package/content/npm" .}}

View File

@ -2418,6 +2418,7 @@
"container", "container",
"debian", "debian",
"generic", "generic",
"go",
"helm", "helm",
"maven", "maven",
"npm", "npm",

View File

@ -0,0 +1,166 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package integration
import (
"archive/zip"
"bytes"
"fmt"
"net/http"
"testing"
"time"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/models/packages"
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/tests"
"github.com/stretchr/testify/assert"
)
func TestPackageGo(t *testing.T) {
defer tests.PrepareTestEnv(t)()
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
packageName := "gitea.com/go-gitea/gitea"
packageVersion := "v0.0.1"
packageVersion2 := "v0.0.2"
goModContent := `module "gitea.com/go-gitea/gitea"`
createArchive := func(files map[string][]byte) []byte {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for name, content := range files {
w, _ := zw.Create(name)
w.Write(content)
}
zw.Close()
return buf.Bytes()
}
url := fmt.Sprintf("/api/packages/%s/go", user.Name)
t.Run("Upload", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
content := createArchive(nil)
req := NewRequestWithBody(t, "PUT", url+"/upload", bytes.NewReader(content))
MakeRequest(t, req, http.StatusUnauthorized)
req = NewRequestWithBody(t, "PUT", url+"/upload", bytes.NewReader(content))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusBadRequest)
content = createArchive(map[string][]byte{
packageName + "@" + packageVersion + "/go.mod": []byte(goModContent),
})
req = NewRequestWithBody(t, "PUT", url+"/upload", bytes.NewReader(content))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusCreated)
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeGo)
assert.NoError(t, err)
assert.Len(t, pvs, 1)
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pvs[0])
assert.NoError(t, err)
assert.Nil(t, pd.Metadata)
assert.Equal(t, packageName, pd.Package.Name)
assert.Equal(t, packageVersion, pd.Version.Version)
pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID)
assert.NoError(t, err)
assert.Len(t, pfs, 1)
assert.Equal(t, packageVersion+".zip", pfs[0].Name)
assert.True(t, pfs[0].IsLead)
pb, err := packages.GetBlobByID(db.DefaultContext, pfs[0].BlobID)
assert.NoError(t, err)
assert.Equal(t, int64(len(content)), pb.Size)
req = NewRequestWithBody(t, "PUT", url+"/upload", bytes.NewReader(content))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusConflict)
time.Sleep(time.Second)
content = createArchive(map[string][]byte{
packageName + "@" + packageVersion2 + "/go.mod": []byte(goModContent),
})
req = NewRequestWithBody(t, "PUT", url+"/upload", bytes.NewReader(content))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusCreated)
})
t.Run("List", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/list", url, packageName))
resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, packageVersion+"\n"+packageVersion2+"\n", resp.Body.String())
})
t.Run("Info", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/%s.info", url, packageName, packageVersion))
resp := MakeRequest(t, req, http.StatusOK)
type Info struct {
Version string `json:"Version"`
Time time.Time `json:"Time"`
}
info := &Info{}
DecodeJSON(t, resp, &info)
assert.Equal(t, packageVersion, info.Version)
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/latest.info", url, packageName))
resp = MakeRequest(t, req, http.StatusOK)
info = &Info{}
DecodeJSON(t, resp, &info)
assert.Equal(t, packageVersion2, info.Version)
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/@latest", url, packageName))
resp = MakeRequest(t, req, http.StatusOK)
info = &Info{}
DecodeJSON(t, resp, &info)
assert.Equal(t, packageVersion2, info.Version)
})
t.Run("GoMod", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/%s.mod", url, packageName, packageVersion))
resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, goModContent, resp.Body.String())
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/latest.mod", url, packageName))
resp = MakeRequest(t, req, http.StatusOK)
assert.Equal(t, goModContent, resp.Body.String())
})
t.Run("Download", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/%s.zip", url, packageName, packageVersion))
MakeRequest(t, req, http.StatusOK)
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/@v/latest.zip", url, packageName))
MakeRequest(t, req, http.StatusOK)
})
}

10
web_src/svg/gitea-go.svg Normal file
View File

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="24.7 77.83 205.42 76.8" xml:space="preserve">
<g style="fill:#00ACD7">
<path d="M40.2,101.1c-0.4,0-0.5-0.2-0.3-0.5l2.1-2.7c0.2-0.3,0.7-0.5,1.1-0.5l35.7,0c0.4,0,0.5,0.3,0.3,0.6 l-1.7,2.6c-0.2,0.3-0.7,0.6-1,0.6L40.2,101.1z"/>
<path d="M25.1,110.3c-0.4,0-0.5-0.2-0.3-0.5l2.1-2.7c0.2-0.3,0.7-0.5,1.1-0.5l45.6,0c0.4,0,0.6,0.3,0.5,0.6 l-0.8,2.4c-0.1,0.4-0.5,0.6-0.9,0.6L25.1,110.3z"/>
<path d="M49.3,119.5c-0.4,0-0.5-0.3-0.3-0.6l1.4-2.5c0.2-0.3,0.6-0.6,1-0.6l20,0c0.4,0,0.6,0.3,0.6,0.7l-0.2,2.4 c0,0.4-0.4,0.7-0.7,0.7L49.3,119.5z"/>
<path d="M153.1,99.3c-6.3,1.6-10.6,2.8-16.8,4.4c-1.5,0.4-1.6,0.5-2.9-1c-1.5-1.7-2.6-2.8-4.7-3.8 c-6.3-3.1-12.4-2.2-18.1,1.5c-6.8,4.4-10.3,10.9-10.2,19c0.1,8,5.6,14.6,13.5,15.7c6.8,0.9,12.5-1.5,17-6.6 c0.9-1.1,1.7-2.3,2.7-3.7c-3.6,0-8.1,0-19.3,0c-2.1,0-2.6-1.3-1.9-3c1.3-3.1,3.7-8.3,5.1-10.9c0.3-0.6,1-1.6,2.5-1.6 c5.1,0,23.9,0,36.4,0c-0.2,2.7-0.2,5.4-0.6,8.1c-1.1,7.2-3.8,13.8-8.2,19.6c-7.2,9.5-16.6,15.4-28.5,17 c-9.8,1.3-18.9-0.6-26.9-6.6c-7.4-5.6-11.6-13-12.7-22.2c-1.3-10.9,1.9-20.7,8.5-29.3c7.1-9.3,16.5-15.2,28-17.3 c9.4-1.7,18.4-0.6,26.5,4.9c5.3,3.5,9.1,8.3,11.6,14.1C154.7,98.5,154.3,99,153.1,99.3z"/>
<path d="M186.2,154.6c-9.1-0.2-17.4-2.8-24.4-8.8c-5.9-5.1-9.6-11.6-10.8-19.3c-1.8-11.3,1.3-21.3,8.1-30.2 c7.3-9.6,16.1-14.6,28-16.7c10.2-1.8,19.8-0.8,28.5,5.1c7.9,5.4,12.8,12.7,14.1,22.3c1.7,13.5-2.2,24.5-11.5,33.9 c-6.6,6.7-14.7,10.9-24,12.8C191.5,154.2,188.8,154.3,186.2,154.6z M210,114.2c-0.1-1.3-0.1-2.3-0.3-3.3 c-1.8-9.9-10.9-15.5-20.4-13.3c-9.3,2.1-15.3,8-17.5,17.4c-1.8,7.8,2,15.7,9.2,18.9c5.5,2.4,11,2.1,16.3-0.6 C205.2,129.2,209.5,122.8,210,114.2z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.7 KiB