Commit Graph

1422 Commits

Author SHA1 Message Date
Lunny Xiao 696fbe6036
Refactor push mirror find and add check for updating push mirror (#32539)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-18 05:59:04 +00:00
Lunny Xiao 4f879a00df
Refactor find forks and fix possible bugs that weak permissions check (#32528)
- Move models/GetForks to services/FindForks
- Add doer as a parameter of FindForks to check permissions
- Slight performance optimization for get forks API with batch loading
of repository units
- Add tests for forking repository to organizations

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-18 03:06:25 +00:00
Lunny Xiao c3dedcffa7
Fix basic auth with webauthn (#32531) 2024-11-16 17:52:16 +00:00
Lunny Xiao e1b269e956
Remove transaction for archive download (#32186)
Since there is a status column in the database, the transaction is
unnecessary when downloading an archive. The transaction is blocking
database operations, especially with SQLite.

Replace #27563
2024-11-15 12:04:20 +08:00
Rowan Bohde 68731c07c5
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
Lunny Xiao 98d9a71ffe
Trim title before insert/update to database to match the size requirements of database (#32498)
Fix #32489
2024-11-14 07:19:14 +00:00
Lunny Xiao b4abb6deff
Reimplement GetUserOrgsList to make it simple and clear (#32486)
Reimplement GetUserOrgsList and also move some functions and test to
org_list file.

---------

Co-authored-by: Zettat123 <zettat123@gmail.com>
2024-11-14 05:31:47 +00:00
wxiaoguang 3f9c3e7bc3
Refactor render system (#32492)
There were too many patches to the Render system, it's really difficult
to make further improvements.

This PR clears the legacy problems and fix TODOs.

1. Rename `RenderContext.Type` to `RenderContext.MarkupType` to clarify
its usage.
2. Use `ContentMode` to replace `meta["mode"]` and `IsWiki`, to clarify
the rendering behaviors.
3. Use "wiki" mode instead of "mode=gfm + wiki=true"
4. Merge `renderByType` and `renderByFile`
5. Add more comments

----

The problem of "mode=document": in many cases it is not set, so many
non-comment places use comment's hard line break incorrectly
2024-11-14 05:02:11 +00:00
wxiaoguang 0aedb03996
Fix LFS route mock, realm, middleware names (#32488)
1. move "internal-lfs" route mock to "common-lfs"
2. fine tune tests
3. fix "realm" strings, according to RFC:
https://datatracker.ietf.org/doc/html/rfc2617:
    * realm       = "realm" "=" realm-value
    * realm-value = quoted-string
4. clarify some names of the middlewares, rename `ignXxx` to `optXxx` to
match `reqXxx`, and rename ambiguous `requireSignIn` to `reqGitSignIn`
2024-11-13 16:58:09 +08:00
Lunny Xiao 840ad7eefe
Disable Oauth check if oauth disabled (#32368)
Fix #32367

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-12 21:33:35 +00:00
6543 43c252dfea
Calculate `PublicOnly` for org membership only once (#32234)
Refactoring of #32211

this move the PublicOnly() filter calcuation next to the DB querys and
let it be decided by the Doer


---
*Sponsored by Kithara Software GmbH*
2024-11-11 01:38:30 +01:00
Zettat123 b1f42a0cdd
Add `DEFAULT_MIRROR_REPO_UNITS` and `DEFAULT_TEMPLATE_REPO_UNITS` options (#32416)
Resolve #30350

The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.

Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
2024-11-11 00:11:00 +00:00
Zettat123 a910abbb45
Add a doctor check to disable the "Actions" unit for mirrors (#32424)
Resolve #32232

Users can disable the "Actions" unit for all mirror repos by running 
```
gitea doctor check --run  disable-mirror-actions-unit --fix
```
2024-11-10 23:37:24 +00:00
wxiaoguang a928739456
Refactor sidebar assignee&milestone&project selectors (#32465)
Follow #32460

Now the code could be much clearer than before and easier to maintain. A
lot of legacy code is removed.

Manually tested.

This PR is large enough, that fine tunes could be deferred to the future if
there is no bug found or design problem.

Screenshots:

<details>

![image](https://github.com/user-attachments/assets/35f4ab7b-1bc0-4bad-a73c-a4569328303c)

</details>
2024-11-11 04:07:54 +08:00
Calvin K 18aeca5320
Add reviewers selection to new pull request (#32403)
Users could add reviewers when creating new PRs.

---------

Co-authored-by: splitt3r <splitt3r@users.noreply.github.com>
Co-authored-by: Sebastian Sauer <sauer.sebastian@gmail.com>
Co-authored-by: bb-ben <70356237+bboerben@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-09 04:48:31 +00:00
Zettat123 35bcd667b2
Fix broken releases when re-pushing tags (#32435)
Fix #32427
2024-11-08 10:53:06 +08:00
Kemal Zebari fb03062478
Only provide the commit summary for Discord webhook push events (#32432)
Resolves #32371.

#31970 should have just showed the commit summary, but
`strings.SplitN()` was misused such that we did not perform any
splitting at all and just used the message. This was not caught in the
unit test made in that PR since the test commit summary was > 50 (which
truncated away the commit description).

This snapshot resolves this and adds another unit test to ensure that we
only show the commit summary.
2024-11-07 19:56:53 +00:00
Lunny Xiao 331e878e81
Add new event commit status creation and webhook implementation (#27151)
This PR introduces a new event which is similar as Github's. When a new
commit status submitted, the event will be trigged. That means, now we
can receive all feedback from CI/CD system in webhooks or other notify
systems.

ref:
https://docs.github.com/en/webhooks/webhook-events-and-payloads#status

Fix #20749
2024-11-07 06:41:49 +00:00
Lunny Xiao 276500c314
Move AddCollabrator and CreateRepositoryByExample to service layer (#32419)
- [x] Move `CreateRepositoryByExample` to service layer
- [x] Move `AddCollabrator` to service layer
- [x] Add a new parameter for `AddCollabrator` so that changing mode
immediately after that will become unnecessary.
2024-11-07 11:28:11 +08:00
Kemal Zebari 7adc4717ec
Include file extension checks in attachment API (#32151)
From testing, I found that issue posters and users with repository write
access are able to edit attachment names in a way that circumvents the
instance-level file extension restrictions using the edit attachment
APIs. This snapshot adds checks for these endpoints.
2024-11-06 21:34:32 +00:00
Lunny Xiao 24b83ff63e
Fix milestone deadline and date related problems (#32339)
Use zero instead of 9999-12-31 for deadline
Fix #32291

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-11-05 07:46:40 +00:00
wxiaoguang 61c35590c7
Refactor RepoRefByType (#32413)
1. clarify the "filepath" could(should) contain "{ref}"
2. remove unclear RepoRefLegacy and RepoRefAny, use RepoRefUnknown to guess
3. by the way, avoid using AppURL
2024-11-05 06:35:54 +00:00
wxiaoguang 4a469c8e1b
Refactor template ctx and render utils (#32422)
Clean up the templates
2024-11-05 14:04:26 +08:00
wxiaoguang b068dbd40e
Refactor DateUtils and merge TimeSince (#32409)
Follow #32383 and #32402
2024-11-04 11:30:00 +00:00
wxiaoguang e524f63d58
Fix git error handling (#32401) 2024-11-02 11:20:22 +00:00
wxiaoguang fec6b3d500
Replace DateTime with DateUtils (#32383) 2024-11-02 04:08:28 +00:00
Rowan Bohde 7dcccc3bb1
improve performance of diffs (#32393)
This has two major changes that significantly reduce the amount of work
done for large diffs:

* Kill a running git process when reaching the maximum number of files
in a diff, preventing it from processing the entire diff.
* When loading a diff with the URL param `file-only=true`, skip loading
stats. This speeds up loading both hidden files of a diff and sections
of a diff when clicking the "Show More" button.

A couple of minor things from profiling are also included:

* Reuse existing repo in `PrepareViewPullInfo` if head and base are the
same.

The performance impact is going to depend heavily on the individual diff
and the hardware it runs on, but when testing locally on a diff changing
100k+ lines over hundreds of files, I'm seeing a roughly 75% reduction
in time to load the result of "Show More"

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-02 11:29:37 +08:00
Zettat123 0690cb076b
Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (#32365)
Fix #28121

I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.

348d1d0f32/routers/api/packages/container/container.go (L712-L717)
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
<img width="600px"
src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>

In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.

```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```

---

Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.

b7fb20e73e/modules/storage/minio.go (L214-L221)
2024-10-31 15:28:25 +00:00
wxiaoguang 5e6523aa57
Update go dependencies (#32389) 2024-10-31 12:05:54 +00:00
Oleksandr Redko f4d3aaeeb9
refactor: remove redundant err declarations (#32381) 2024-10-30 19:36:24 +00:00
Royce Remer c60e4dc109
Add new [lfs_client].BATCH_SIZE and [server].LFS_MAX_BATCH_SIZE config settings. (#32307)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'

This fixes #32306

---------

Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-10-30 05:41:55 +00:00
wxiaoguang a920fcfd91
Fix db engine (#32351)
Fix #32349
2024-10-27 18:48:07 -04:00
wxiaoguang d70af38447
Refactor the DB migration system slightly (#32344)
Introduce "idNumber" for each migration, and clarify the difference
between the migration ID number and database version.
2024-10-27 19:54:35 +08:00
Tim de2ad2e1b1
Make admins adhere to branch protection rules (#32248)
This introduces a new flag `BlockAdminMergeOverride` on the branch
protection rules that prevents admins/repo owners from bypassing branch
protection rules and merging without approvals or failing status checks.

Fixes #17131

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-10-23 12:39:43 +08:00
Lunny Xiao a264c46fb0
Add warn log when deleting inactive users (#32318)
Add log for the problem #31480
2024-10-23 09:28:28 +08:00
谈笑风生间 d50ed0abf7
Support requested_reviewers data in comment webhook events (#26178)
close #25833 

Currently, the information for "requested_reviewers" is only included in
the webhook event for reviews. I would like to suggest adding this
information to the webhook event for "PullRequest comment" as well, as
they both pertain to the "PullRequest" event.

Also, The reviewer information for the Pull Request is not displayed
when it is approved or rejected.
2024-10-16 09:10:05 +00:00
Zettat123 81aec6d621
Update scheduled tasks even if changes are pushed by "ActionsUser" (#32246)
Fix #32219

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-10-13 20:28:32 +00:00
Zettat123 74664b08a0
Support migrating GitHub/GitLab PR draft status (#32242)
Resolve #32196
2024-10-13 22:58:13 +03:00
Lunny Xiao 9df5ddaf44
Only rename a user when they should receive a different name (#32247)
Fix #31996
2024-10-13 03:13:55 +00:00
wxiaoguang afa8dd45af
Make git push options accept short name (#32245)
Just like what most CLI parsers do: `--opt` means `opt=true`

Then users could use `-o force-push` as `-o force-push=true`
2024-10-12 05:42:10 +00:00
Zettat123 0fe5e2b08c
Allow maintainers to view and edit files of private repos when "Allow maintainers to edit" is enabled (#32215)
Fix #31539
2024-10-11 19:08:19 +00:00
wxiaoguang 6029d78ab5
Improve the maintainblity of the reserved username list (#32229) 2024-10-10 09:04:42 +00:00
wxiaoguang dd83cfcacc
Refactor CSRF token (#32216) 2024-10-10 03:48:21 +00:00
Lunny Xiao d6d3c96e65
Fix bug when a token is given public only (#32204) 2024-10-08 12:51:09 +03:00
Lunny Xiao a989404e23
Fix javascript error when an anonymous user visiting migration page (#32144)
This PR fixes javascript errors when an anonymous user visits the
migration page.
It also makes task view checking more restrictive.

The router moved from `/user/task/{id}/status` to
`/username/reponame/-/migrate/status` because it's a migrate status.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-10-02 04:37:16 +00:00
Lunny Xiao 3a4a1bffbe
Make oauth2 code clear. Move oauth2 provider code to their own packages/files (#32148)
Fix #30266
Replace #31533
2024-10-02 08:03:19 +08:00
yp05327 70b7df0e5e
Support repo license (#24872)
Close #278
Close #24076

## Solutions:
- Use
[google/licenseclassifier](https://github.com/google/licenseclassifier/)
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[go-license-detector](https://github.com/go-enry/go-license-detector):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[google/licenseclassifier](https://github.com/google/licenseclassifier/):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178
- Generate License Convert Name List to avoid import license templates
with same contents
Gitea automatically get latest license data from[
spdx/license-list-data](https://github.com/spdx/license-list-data).
But unfortunately, some license templates have same contents. #20915
[click here to see the
list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684)
So we will generate a list of these license templates with same contents
and create a new file to save the result when using `make
generate-license`. (Need to decide the save path)
- Save License info into a new table `repo_license`
Can easily support searching repo by license in the future.

## Screen shot
Single License:

![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f)

Multiple Licenses:

![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70)

Triggers:
- [x] Push commit to default branch
- [x] Create repo
- [x] Mirror repo
- [x] When Default Branch is changed, licenses should be updated

Todo:
- [x] Save Licenses info in to DB when there's a change to license file
in the commit
- [x] DB Migration
- [x] A nominal test?
- [x] Select which library to
use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167)
- [x] API Support
- [x] Add repo license table
- ~Select license in settings if there are several licenses(Not
recommended)~
- License board(later, not in this PR)

![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.com>
2024-10-01 15:25:08 -04:00
Zettat123 f4b8f6fc40
Fix the logic of finding the latest pull review commit ID (#32139)
Fix #31423
2024-10-01 01:58:55 +00:00
Lunny Xiao 5a8568459d
Fix bug when deleting a migrated branch (#32075)
After migrating a repository with pull request, the branch is missed and
after the pull request merged, the branch cannot be deleted.
2024-09-24 07:42:08 +00:00
Kemal Zebari aadbe0488f
Truncate commit message during Discord webhook push events (#31970)
Resolves #31668.
2024-09-24 03:38:08 +00:00