Commit Graph

290 Commits

Author SHA1 Message Date
Lunny Xiao f62f68cbdd
Move team related functions to service layer (#32537)
There are still some functions under `models` after last big refactor
about `models`. This change will move all team related functions to
service layer with no code change.
2024-11-27 21:12:26 +00:00
Lunny Xiao 703be6bf30
Add github compatible tarball download API endpoints (#32572)
Fix #29654 
Fix #32481
2024-11-25 19:35:49 +00:00
Lunny Xiao fe49cb0243
Fix get reviewers' bug (#32415)
This PR rewrites `GetReviewer` function and move it to service layer.

Reviewers should not be watchers, so that this PR removed all watchers
from reviewers. When the repository is under an organization, the pull
request unit read permission will be checked to resolve the bug of
#32394

Fix #32394
2024-11-22 10:44:48 -05:00
Rowan Bohde 9ac74a1a40
disable gravatar in test (#32529)
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
2024-11-21 04:30:48 +00:00
Lunny Xiao 4f879a00df
Refactor find forks and fix possible bugs that weak permissions check (#32528)
- Move models/GetForks to services/FindForks
- Add doer as a parameter of FindForks to check permissions
- Slight performance optimization for get forks API with batch loading
of repository units
- Add tests for forking repository to organizations

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-18 03:06:25 +00:00
Lunny Xiao e1b269e956
Remove transaction for archive download (#32186)
Since there is a status column in the database, the transaction is
unnecessary when downloading an archive. The transaction is blocking
database operations, especially with SQLite.

Replace #27563
2024-11-15 12:04:20 +08:00
Rowan Bohde 68731c07c5
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
Zettat123 b1f42a0cdd
Add `DEFAULT_MIRROR_REPO_UNITS` and `DEFAULT_TEMPLATE_REPO_UNITS` options (#32416)
Resolve #30350

The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.

Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
2024-11-11 00:11:00 +00:00
Zettat123 35bcd667b2
Fix broken releases when re-pushing tags (#32435)
Fix #32427
2024-11-08 10:53:06 +08:00
Lunny Xiao 331e878e81
Add new event commit status creation and webhook implementation (#27151)
This PR introduces a new event which is similar as Github's. When a new
commit status submitted, the event will be trigged. That means, now we
can receive all feedback from CI/CD system in webhooks or other notify
systems.

ref:
https://docs.github.com/en/webhooks/webhook-events-and-payloads#status

Fix #20749
2024-11-07 06:41:49 +00:00
Lunny Xiao 276500c314
Move AddCollabrator and CreateRepositoryByExample to service layer (#32419)
- [x] Move `CreateRepositoryByExample` to service layer
- [x] Move `AddCollabrator` to service layer
- [x] Add a new parameter for `AddCollabrator` so that changing mode
immediately after that will become unnecessary.
2024-11-07 11:28:11 +08:00
wxiaoguang e524f63d58
Fix git error handling (#32401) 2024-11-02 11:20:22 +00:00
Rowan Bohde 7dcccc3bb1
improve performance of diffs (#32393)
This has two major changes that significantly reduce the amount of work
done for large diffs:

* Kill a running git process when reaching the maximum number of files
in a diff, preventing it from processing the entire diff.
* When loading a diff with the URL param `file-only=true`, skip loading
stats. This speeds up loading both hidden files of a diff and sections
of a diff when clicking the "Show More" button.

A couple of minor things from profiling are also included:

* Reuse existing repo in `PrepareViewPullInfo` if head and base are the
same.

The performance impact is going to depend heavily on the individual diff
and the hardware it runs on, but when testing locally on a diff changing
100k+ lines over hundreds of files, I'm seeing a roughly 75% reduction
in time to load the result of "Show More"

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-02 11:29:37 +08:00
yp05327 70b7df0e5e
Support repo license (#24872)
Close #278
Close #24076

## Solutions:
- Use
[google/licenseclassifier](https://github.com/google/licenseclassifier/)
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[go-license-detector](https://github.com/go-enry/go-license-detector):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[google/licenseclassifier](https://github.com/google/licenseclassifier/):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178
- Generate License Convert Name List to avoid import license templates
with same contents
Gitea automatically get latest license data from[
spdx/license-list-data](https://github.com/spdx/license-list-data).
But unfortunately, some license templates have same contents. #20915
[click here to see the
list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684)
So we will generate a list of these license templates with same contents
and create a new file to save the result when using `make
generate-license`. (Need to decide the save path)
- Save License info into a new table `repo_license`
Can easily support searching repo by license in the future.

## Screen shot
Single License:

![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f)

Multiple Licenses:

![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70)

Triggers:
- [x] Push commit to default branch
- [x] Create repo
- [x] Mirror repo
- [x] When Default Branch is changed, licenses should be updated

Todo:
- [x] Save Licenses info in to DB when there's a change to license file
in the commit
- [x] DB Migration
- [x] A nominal test?
- [x] Select which library to
use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167)
- [x] API Support
- [x] Add repo license table
- ~Select license in settings if there are several licenses(Not
recommended)~
- License board(later, not in this PR)

![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.com>
2024-10-01 15:25:08 -04:00
Lunny Xiao 5a8568459d
Fix bug when deleting a migrated branch (#32075)
After migrating a repository with pull request, the branch is missed and
after the pull request merged, the branch cannot be deleted.
2024-09-24 07:42:08 +00:00
Exploding Dragon 269c630923
Fix: database not update release when using `git push --tags --force` (#32040)
link: https://codeberg.org/forgejo/forgejo/issues/4274

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2024-09-18 20:15:03 +00:00
Lunny Xiao 2da2000413
Use global lock instead of NewExclusivePool to allow distributed lock between multiple Gitea instances (#31813)
Replace #26486 
Fix #19620

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2024-09-06 10:12:41 +00:00
Edip Emre Bodur a4dac59643
Fixes for unreachable project issues when transfer repository from organization (#31770)
When transferring repositories that have issues linked to a project
board to another organization, the issues remain associated with the
original project board. This causes the columns in the project board to
become bugged, making it difficult to move other issues in or out of the
affected columns. As a solution, I removed the issue relations since the
other organization does not have this project table.

Fix for #31538

Co-authored-by: Jason Song <i@wolfogre.com>
2024-08-13 02:53:43 +00:00
Fábio Barkoski e45a4c9829
Move repository visibility to danger zone in the settings area (#31126)
Moved repository visibility to the danger zone in the settings area. To
change the visibility, it is necessary to go to the danger zone, click
on the private/public button, and accept the change in the modal.

Resolves:  #23826

---
## Screenshots

<details>
<summary>Before</summary>
Private repo:

![Private
repo](https://github.com/go-gitea/gitea/assets/65479069/4313492a-4854-48bc-9f47-974e3539d791)

Public repo:

![Public
repo](https://github.com/go-gitea/gitea/assets/65479069/1c45f6e4-ee93-4799-9331-e9d4a7e0f16a)

</details>
<details>
<summary>After</summary>
Make private:

![Screenshot from 2024-05-28
21-35-38](https://github.com/go-gitea/gitea/assets/65479069/4887e28a-0514-4990-aa69-bf3ddc7e6c7d)

Make private modal

![Screenshot from 2024-06-13
23-55-55](https://github.com/go-gitea/gitea/assets/65479069/9f5a7604-069b-41a2-973b-ee2d58e85953)

![Screenshot from 2024-06-13
23-53-09](https://github.com/go-gitea/gitea/assets/65479069/06c22726-eab2-4bce-8df7-62849dcce974)

Make public:

![Screenshot from 2024-05-28
21-34-27](https://github.com/go-gitea/gitea/assets/65479069/6d388f99-0356-48a0-9d85-320cdba55179)

Make public modal

![Screenshot from 2024-06-13
23-53-37](https://github.com/go-gitea/gitea/assets/65479069/8944972e-f2d4-4aea-ba96-b892febb5ced)

</details>

---------

Co-authored-by: Kemal Zebari <60799661+kemzeb@users.noreply.github.com>
2024-08-11 07:50:54 +03:00
Jason Song 09b56fc069
Distinguish LFS object errors to ignore missing objects during migration (#31702)
Fix #31137.

Replace #31623 #31697.

When migrating LFS objects, if there's any object that failed (like some
objects are losted, which is not really critical), Gitea will stop
migrating LFS immediately but treat the migration as successful.

This PR checks the error according to the [LFS api
doc](https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md#successful-responses).

> LFS object error codes should match HTTP status codes where possible:
> 
> - 404 - The object does not exist on the server.
> - 409 - The specified hash algorithm disagrees with the server's
acceptable options.
> - 410 - The object was removed by the owner.
> - 422 - Validation error.

If the error is `404`, it's safe to ignore it and continue migration.
Otherwise, stop the migration and mark it as failed to ensure data
integrity of LFS objects.

And maybe we should also ignore others errors (maybe `410`? I'm not sure
what's the difference between "does not exist" and "removed by the
owner".), we can add it later when some users report that they have
failed to migrate LFS because of an error which should be ignored.
2024-07-31 10:29:48 +00:00
Zettat123 e9aa39bda4
Fix a branch divergence cache bug (#31659)
Fix #31599

A branch divergence is counted based on the default branch. If the
default branch is updated, all divergence caches of the repo need to be
deleted.
2024-07-19 17:52:49 +02:00
delvh 7bb3835811
Add `YEAR`, `MONTH`, `MONTH_ENGLISH`, `DAY` variables for template repos (#31584)
Fixes #31583
2024-07-09 18:05:12 +00:00
wxiaoguang 43c7a2e7b1
Refactor names (#31405)
This PR only does "renaming":

* `Route` should be `Router` (and chi router is also called "router")
* `Params` should be `PathParam` (to distingush it from URL query param, and to match `FormString`)
* Use lower case for private functions to avoid exposing or abusing
2024-06-19 06:32:45 +08:00
wxiaoguang f5dfd7d73c
Add a simple test for AdoptRepository (#31391)
Follow #31333
2024-06-17 01:18:35 +00:00
silverwind 816222243a
Add `lint-go-gopls` (#30729)
Uses `gopls check <files>` as a linter. Tested locally and brings up 149
errors currently for me. I don't think I want to fix them in this PR,
but I would like at least to get this analysis running on CI.

List of errors:
```
modules/indexer/code/indexer.go:181:11: impossible condition: nil != nil
routers/private/hook_post_receive.go:120:15: tautological condition: nil == nil
services/auth/source/oauth2/providers.go:185:9: tautological condition: nil == nil
services/convert/issue.go:216:11: tautological condition: non-nil != nil
tests/integration/git_test.go:332:9: impossible condition: nil != nil
services/migrations/migrate.go:179:24-43: unused parameter: ctx
services/repository/transfer.go:288:48-69: unused parameter: doer
tests/integration/api_repo_tags_test.go:75:41-61: unused parameter: session
tests/integration/git_test.go:696:64-74: unused parameter: baseBranch
tests/integration/gpg_git_test.go:265:27-39: unused parameter: t
tests/integration/gpg_git_test.go:284:23-29: unused parameter: tmpDir
tests/integration/gpg_git_test.go:284:31-35: unused parameter: name
tests/integration/gpg_git_test.go:284:37-42: unused parameter: email
```
2024-06-05 09:22:38 +08:00
Lunny Xiao 5c1b550e00
Fix push multiple branches error with tests (#31151) 2024-05-29 06:43:02 +00:00
Lunny Xiao c6cf96d31d
Fix automerge will not work because of some events haven't been triggered (#30780)
Replace #25741
Close #24445
Close #30658
Close #20646
~Depends on #30805~

Since #25741 has been rewritten totally, to make the contribution
easier, I will continue the work in this PR. Thanks @6543

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-05-21 23:23:22 +08:00
wxiaoguang edbf74c418
Fix "force private" logic (#31012)
When creating a repo, the "FORCE_PRIVATE" config option should be
respected, `readonly` doesn't work for checkbox, so it should use
`disabled` attribute.
2024-05-20 00:56:45 +00:00
yp05327 e94723f2de
Fix incorrect default branch when adopt a repository (#30912)
Fix #30521

we should sync branches first, then detect default branch, or
`git_model.FindBranchNames` will always return empty list, and the
detection will be wrong.
2024-05-09 08:44:26 +00:00
wxiaoguang 9c08637eae
Make "sync branch" also sync object format and add tests (#30878) 2024-05-06 17:02:30 +00:00
wxiaoguang 7c613f100e
Make sure git version&feature are always prepared (#30877)
Otherwise there would be more similar issues like #29287
2024-05-06 18:34:16 +02:00
Chongyi Zheng e80466f734
Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Lunny Xiao c685eefe4a
If a repository return no commitstatus, then still cache it but not query it from database (#30700)
The previous repository default branch commit status cache will only
store if the commit status has value. So the repository which have no
any commit status will always be fetched from database.

This PR will store the empty state of commit status of a repository into
cache because the cache will be updated once there is a commit status
stored.
2024-04-25 17:14:23 +08:00
silverwind 74f0c84fa4
Enable more `revive` linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
wxiaoguang 7cb88e9c18
Use correct hash for "git update-index" (#30626) 2024-04-22 00:13:44 +00:00
wxiaoguang 48d4580dd5
Clarify permission "HasAccess" behavior (#30585)
Follow #30495

"HasAccess" behavior wasn't clear, to make it clear:

* Use a new name `HasAnyUnitAccess`, it will be easier to review related
code and permission problems.
* Separate everyone access mode to a separate field, then all calls to
HasAccess are reverted to old behavior before #30495.
* Add new tests.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-20 03:15:04 +00:00
wxiaoguang c248f010ad
Refactor cache and disable go-chi cache (#30417)
use built-in cache package to wrap external go-chi cache package
2024-04-13 08:38:44 +00:00
Lunny Xiao f9c3e79aba
Fix commit status cache which missed target_url (#30426)
Fix #30421

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2024-04-12 11:02:42 +00:00
Lunny Xiao fc34481d05
Add commit status summary table to reduce query from commit status table (#30223)
This PR adds a new table named commit status summary to reduce queries
from the commit status table. After this change, commit status summary
table will be used for the final result, commit status table will be for
details.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2024-04-12 09:41:50 +08:00
Zettat123 0fe9f93eb4
Check the token's owner and repository when registering a runner (#30406)
Fix #30378
2024-04-11 16:01:44 +08:00
wxiaoguang 83f83019ef
Clean up log messages (#30313)
`log.Xxx("%v")` is not ideal, this PR adds necessary context messages.
Remove some unnecessary logs.

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-07 19:17:06 +08:00
Lunny Xiao f8ab9dafb7
Use db.ListOptionsAll instead of db.ListOptions{ListAll: true} (#29995) 2024-03-22 13:53:52 +01:00
Jason Song b150ff0bab
Cancel previous runs of the same PR automatically (#29961)
Follow #25716. Also cancel previous runs for `pull_request_sync`.

It's not a bug since it original PR said "if the event is push".

The main change is
https://github.com/go-gitea/gitea/pull/29961/files#diff-08adda3f8ae0360937f46abb1f4418603bd3518522baa356be11c6c7ac4abcc3.

And also rename `CancelRunningJobs` to `CancelPreviousJobs` to make it
more clear.
2024-03-21 15:01:35 +08:00
wxiaoguang 21151474e3
Fix loadOneBranch panic (#29938)
Try to fix #29936

Far from ideal, but still better than panic.
2024-03-20 15:38:22 +00:00
Jason Song f371f84fa3
Restore deleted branches when syncing (#29898)
Regression of #29493. If a branch has been deleted, repushing it won't
restore it.

Lunny may have noticed that, but I didn't delve into the comment then
overlooked it:
https://github.com/go-gitea/gitea/pull/29493#discussion_r1509046867

The additional comments added are to explain the issue I found during
testing, which are unrelated to the fixes.
2024-03-20 01:45:27 +00:00
Lunny Xiao 930bae2300
Add cache for branch divergence on branch list page (#29577)
The branch page for blender project will take 6s because calculating
divergence is very slow.
This PR will add a cache for the branch divergence calculation. So when
the second visit the branch list, it will take only less 200ms.
2024-03-08 10:21:24 +00:00
Lunny Xiao 25b842df26
Move get/set default branch from git package to gitrepo package to hide repopath (#29126) 2024-03-08 15:30:10 +08:00
Earl Warren f6d01ac2d8
Add download URL for executable files (#28260)
Consider executable files as a valid case when returning a DownloadURL for them.
They are just regular files with the difference being the executable permission bit being set.

Co-authored-by: Gusted <postmaster@gusted.xyz>
2024-03-06 16:10:06 +00:00
Lunny Xiao e308d25f1b
Cache repository default branch commit status to reduce query on commit status table (#29444)
After repository commit status has been introduced on dashaboard, the
most top SQL comes from `GetLatestCommitStatusForPairs`.

This PR adds a cache for the repository's default branch's latest
combined commit status. When a new commit status updated, the cache will
be marked as invalid.

<img width="998" alt="image"
src="https://github.com/go-gitea/gitea/assets/81045/76759de7-3a83-4d54-8571-278f5422aed3">
2024-03-06 12:17:19 +00:00
Jason Song 5bdf805e05
Sync branches to DB immediately when handle git hook calling (#29493)
Unlike other async processing in the queue, we should sync branches to
the DB immediately when handling git hook calling. If it fails, users
can see the error message in the output of the git command.

It can avoid potential inconsistency issues, and help #29494.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2024-03-06 16:47:52 +08:00