Commit Graph

4623 Commits

Author SHA1 Message Date
Albin Hedman 2763766f85
cargo registry - respect renamed dependencies (#32430)
rust allows renaming dependencies such as when depending on multiple
versions of the same package. This is not supported by gitea as
discovered in #31500 . This PR tries to address that.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-12 02:57:30 +00:00
wxiaoguang 580e21dd2e
Refactor LFS SSH and internal routers (#32473)
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:

* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)

Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
2024-11-12 02:38:22 +00:00
Zettat123 b1f42a0cdd
Add `DEFAULT_MIRROR_REPO_UNITS` and `DEFAULT_TEMPLATE_REPO_UNITS` options (#32416)
Resolve #30350

The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.

Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
2024-11-11 00:11:00 +00:00
wxiaoguang a928739456
Refactor sidebar assignee&milestone&project selectors (#32465)
Follow #32460

Now the code could be much clearer than before and easier to maintain. A
lot of legacy code is removed.

Manually tested.

This PR is large enough, that fine tunes could be deferred to the future if
there is no bug found or design problem.

Screenshots:

<details>

![image](https://github.com/user-attachments/assets/35f4ab7b-1bc0-4bad-a73c-a4569328303c)

</details>
2024-11-11 04:07:54 +08:00
Calvin K 18aeca5320
Add reviewers selection to new pull request (#32403)
Users could add reviewers when creating new PRs.

---------

Co-authored-by: splitt3r <splitt3r@users.noreply.github.com>
Co-authored-by: Sebastian Sauer <sauer.sebastian@gmail.com>
Co-authored-by: bb-ben <70356237+bboerben@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-09 04:48:31 +00:00
Lunny Xiao 331e878e81
Add new event commit status creation and webhook implementation (#27151)
This PR introduces a new event which is similar as Github's. When a new
commit status submitted, the event will be trigged. That means, now we
can receive all feedback from CI/CD system in webhooks or other notify
systems.

ref:
https://docs.github.com/en/webhooks/webhook-events-and-payloads#status

Fix #20749
2024-11-07 06:41:49 +00:00
wxiaoguang 145e266987
Support quote selected comments to reply (#32431)
Many existing tests were quite hacky, these could be improved later.

<details>

![image](https://github.com/user-attachments/assets/93aebb4f-9de5-4cb8-910b-50c64cbcd25a)

</details>
2024-11-07 03:57:07 +00:00
Lunny Xiao 276500c314
Move AddCollabrator and CreateRepositoryByExample to service layer (#32419)
- [x] Move `CreateRepositoryByExample` to service layer
- [x] Move `AddCollabrator` to service layer
- [x] Add a new parameter for `AddCollabrator` so that changing mode
immediately after that will become unnecessary.
2024-11-07 11:28:11 +08:00
Bruno Sofiato f64fbd9b74
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-11-06 20:51:20 +00:00
wxiaoguang 41b4ef825d
Use 8 as default value for git lfs concurrency (#32421) 2024-11-05 13:10:57 +00:00
Lunny Xiao 24b83ff63e
Fix milestone deadline and date related problems (#32339)
Use zero instead of 9999-12-31 for deadline
Fix #32291

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-11-05 07:46:40 +00:00
wxiaoguang 4a469c8e1b
Refactor template ctx and render utils (#32422)
Clean up the templates
2024-11-05 14:04:26 +08:00
wxiaoguang b068dbd40e
Refactor DateUtils and merge TimeSince (#32409)
Follow #32383 and #32402
2024-11-04 11:30:00 +00:00
wxiaoguang 61be51e56b
Refactor markup package (#32399)
To make the markup package easier to maintain:
1. Split some go files into small files
2. Use a shared util.NopCloser, remove duplicate code
3. Remove unused functions
2024-11-04 10:59:50 +00:00
Royce Remer 54146e62c0
Make LFS http_client parallel within a batch. (#32369)
Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-04 04:49:08 +00:00
wxiaoguang 259811617b
Replace DateTime with proper functions (#32402)
Follow #32383

This PR cleans up the "Deadline" usages in templates, make them call
`ParseLegacy` first to get a `Time` struct then display by `DateUtils`.

Now it should be pretty clear how "deadline string" works, it makes it
possible to do further refactoring and correcting.
2024-11-02 21:04:53 +00:00
wxiaoguang e524f63d58
Fix git error handling (#32401) 2024-11-02 11:20:22 +00:00
Lunny Xiao 13a203828c
Fix created_unix for mirroring (#32342)
Fix #32233
2024-11-02 06:11:38 +00:00
wxiaoguang fec6b3d500
Replace DateTime with DateUtils (#32383) 2024-11-02 04:08:28 +00:00
wxiaoguang ec2d1593c2
Refactor tests to prevent from unnecessary preparations (#32398) 2024-11-01 23:18:29 +08:00
Kyle D. 66971e591e
Add artifacts test fixture (#30300)
Closes https://github.com/go-gitea/gitea/issues/30296

- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
2024-11-01 10:29:54 +08:00
Zettat123 0690cb076b
Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (#32365)
Fix #28121

I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.

348d1d0f32/routers/api/packages/container/container.go (L712-L717)
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
<img width="600px"
src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>

In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.

```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```

---

Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.

b7fb20e73e/modules/storage/minio.go (L214-L221)
2024-10-31 15:28:25 +00:00
wxiaoguang 5e6523aa57
Update go dependencies (#32389) 2024-10-31 12:05:54 +00:00
Oleksandr Redko f4d3aaeeb9
refactor: remove redundant err declarations (#32381) 2024-10-30 19:36:24 +00:00
Royce Remer c60e4dc109
Add new [lfs_client].BATCH_SIZE and [server].LFS_MAX_BATCH_SIZE config settings. (#32307)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'

This fixes #32306

---------

Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-10-30 05:41:55 +00:00
Lunny Xiao feca8802b8
Fix clean tmp dir (#32360)
Try to fix #31792 

Credit to @jeroenlaylo
Copied from
https://github.com/go-gitea/gitea/issues/31792#issuecomment-2311920520

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-10-29 18:27:03 +02:00
Tim de2ad2e1b1
Make admins adhere to branch protection rules (#32248)
This introduces a new flag `BlockAdminMergeOverride` on the branch
protection rules that prevents admins/repo owners from bypassing branch
protection rules and merging without approvals or failing status checks.

Fixes #17131

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-10-23 12:39:43 +08:00
Zettat123 9206fbb55f
Add `DISABLE_ORGANIZATIONS_PAGE` and `DISABLE_CODE_PAGE` settings for explore pages and fix an issue related to user search (#32288)
These settings can allow users to only display the repositories explore page.

Thanks to yp05327 and wxiaoguang !

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-10-22 13:09:19 +08:00
谈笑风生间 d50ed0abf7
Support requested_reviewers data in comment webhook events (#26178)
close #25833 

Currently, the information for "requested_reviewers" is only included in
the webhook event for reviews. I would like to suggest adding this
information to the webhook event for "PullRequest comment" as well, as
they both pertain to the "PullRequest" event.

Also, The reviewer information for the Pull Request is not displayed
when it is approved or rejected.
2024-10-16 09:10:05 +00:00
Zettat123 74664b08a0
Support migrating GitHub/GitLab PR draft status (#32242)
Resolve #32196
2024-10-13 22:58:13 +03:00
wxiaoguang afa8dd45af
Make git push options accept short name (#32245)
Just like what most CLI parsers do: `--opt` means `opt=true`

Then users could use `-o force-push` as `-o force-push=true`
2024-10-12 05:42:10 +00:00
Bruno Sofiato 900ac62251
Allow code search by filename (#32210)
This is a large and complex PR, so let me explain in detail its changes.

First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).

I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.

I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.

The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).


![image](https://github.com/user-attachments/assets/9d938d86-1a8d-4f89-8644-1921a473e858)

Resolves #32096

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-11 23:35:04 +00:00
Ehsan Shirvanian 8bee7fcf7e
update git book link to v2 (#32221)
Fix the dead link `https://git-scm.com/book/en/Git-Basics-Getting-a-Git-Repository` for
empty repositories to help how to clone the repository
to `https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository` 
which is v2 of the git book. This also updates download git links
2024-10-09 13:04:34 +08:00
Zisu Zhang 66923e02d2
Enhance USER_DISABLED_FEATURES to allow disabling change username or full name (#31959)
Fix #31958
Enhanced `USER_DISABLED_FEATURES`(also `EXTERNAL_USER_DISABLE_FEATURES`)
option in `[admin]` section.
Added following values:
- `change_username`: Disable change username
- `change_full_name`: Disable change full name
---

Progress:
- [x] Update code
- [x] Update translations
2024-10-05 20:41:38 +00:00
Bruno Sofiato d266d190bd
Fixed race condition when deleting documents by repoId in ElasticSearch (#32185)
Resolves #32184

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-03 16:03:36 +00:00
yp05327 70b7df0e5e
Support repo license (#24872)
Close #278
Close #24076

## Solutions:
- Use
[google/licenseclassifier](https://github.com/google/licenseclassifier/)
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[go-license-detector](https://github.com/go-enry/go-license-detector):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[google/licenseclassifier](https://github.com/google/licenseclassifier/):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178
- Generate License Convert Name List to avoid import license templates
with same contents
Gitea automatically get latest license data from[
spdx/license-list-data](https://github.com/spdx/license-list-data).
But unfortunately, some license templates have same contents. #20915
[click here to see the
list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684)
So we will generate a list of these license templates with same contents
and create a new file to save the result when using `make
generate-license`. (Need to decide the save path)
- Save License info into a new table `repo_license`
Can easily support searching repo by license in the future.

## Screen shot
Single License:

![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f)

Multiple Licenses:

![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70)

Triggers:
- [x] Push commit to default branch
- [x] Create repo
- [x] Mirror repo
- [x] When Default Branch is changed, licenses should be updated

Todo:
- [x] Save Licenses info in to DB when there's a change to license file
in the commit
- [x] DB Migration
- [x] A nominal test?
- [x] Select which library to
use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167)
- [x] API Support
- [x] Add repo license table
- ~Select license in settings if there are several licenses(Not
recommended)~
- License board(later, not in this PR)

![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.com>
2024-10-01 15:25:08 -04:00
Bruno Sofiato 99d0510cb6
Change the code search to sort results by relevance (#32134)
Resolves #32129

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-09-28 20:13:55 +00:00
ConcurrentCrab 8a9fd7f771
Add pure SSH LFS support (#31516)
Fixes #17554
/claim #17554

Docs PR https://gitea.com/gitea/docs/pulls/49

To test, run pushes like: `GIT_TRACE=1` git push. The trace output
should mention "pure SSH connection".
2024-09-27 10:27:37 -04:00
yp05327 6fa962f409
Fix wrong status of `Set up Job` when first step is skipped (#32120)
Fix #32089
2024-09-24 18:34:08 +00:00
Jamie Schouten d351a42494
Add bin to Composer Metadata (#32099)
This PR addresses the missing `bin` field in Composer metadata, which
currently causes vendor-provided binaries to not be symlinked to
`vendor/bin` during installation.

In the current implementation, running `composer install` does not
publish the binaries, leading to issues where expected binaries are not
available.

By properly declaring the `bin` field, this PR ensures that binaries are
correctly symlinked upon installation, as described in the [Composer
documentation](https://getcomposer.org/doc/articles/vendor-binaries.md).
2024-09-21 22:42:17 +00:00
Lunny Xiao a802508f88
Fix wrong last modify time (#32102) 2024-09-21 21:56:25 +00:00
Lunny Xiao 8e2dd5d3dd
Use camo.Always instead of camo.Allways (#32097)
Fix #31575 

https://gitea.com/gitea/docs/pulls/73
2024-09-21 12:50:54 +03:00
wxiaoguang 3b10fd9b34
Only use Host header from reverse proxy (#32060)
X-Forwarded-Host has many problems: non-standard, not well-defined
(X-Forwarded-Port or not), conflicts with Host header, it already caused
problems like #31907. So do not use X-Forwarded-Host, just use Host
header directly.

Official document also only uses `Host` header and never mentioned
others.
2024-09-20 14:57:55 +00:00
hiifong f38e101448
Lazy load avatar images (#32051) 2024-09-17 19:02:48 +00:00
Jason Song 125679f2e1
Support allowed hosts for migrations to work with proxy (#32025) 2024-09-11 05:47:00 +00:00
Zettat123 def1c9670b
Support migration from AWS CodeCommit (#31981)
This PR adds support for migrating repos from [AWS
CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/welcome.html).

The access key ID and secret access key are required to get repository
information and pull requests. And [HTTPS Git
credentials](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html)
are required to clone the repository.

<img
src="https://github.com/user-attachments/assets/82ecb2d0-8d43-42b0-b5af-f5347a13b9d0"
width="680" />

The AWS CodeCommit icon is from [AWS Architecture
Icons](https://aws.amazon.com/architecture/icons/).

<img
src="https://github.com/user-attachments/assets/3c44d21f-d753-40f5-9eae-5d3589e0d50d"
width="320" />
2024-09-11 07:49:42 +08:00
techknowlogick d9a7748cdc
bump to go 1.23 (#31855) 2024-09-10 02:23:07 +00:00
Jason Song 1620e3f03b
Enable compression for Actions logs by default (#32013)
Close #31801. Follow #31761.

Since there are so many benefits of compression and there are no reports
of related issues after weeks, it should be fine to enable compression
by default.
2024-09-09 21:34:42 +00:00
techknowlogick f183783baa
Save initial signup information for users to aid in spam prevention (#31852)
This will allow instance admins to view signup pattern patterns for
public instances. It is modelled after discourse, mastodon, and
MediaWiki's approaches.

Note: This has privacy implications, but as the above-stated open-source
projects take this approach, especially MediaWiki, which I have no doubt
looked into this thoroughly, it is likely okay for us, too. However, I
would be appreciative of any feedback on how this could be improved.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-09-09 17:05:16 -04:00
Jason Song a323a82ec4
Increase `cacheContextLifetime` to reduce false reports (#32011)
Replace #32001.

To prevent the context cache from being misused for long-term work
(which would result in using invalid cache without awareness), the
context cache is designed to exist for a maximum of 10 seconds. This
leads to many false reports, especially in the case of slow SQL.

This PR increases it to 5 minutes to reduce false reports.

5 minutes is not a very safe value, as a lot of changes may have
occurred within that time frame. However, as far as I know, there has
not been a case of misuse of context cache discovered so far, so I think
5 minutes should be OK.

Please note that after this PR, if warning logs are found again, it
should get attention, at that time it can be almost 100% certain that it
is a misuse.
2024-09-09 04:51:25 +00:00