Commit Graph

511 Commits

Author SHA1 Message Date
wxiaoguang b01b0b99a5
Refactor some LDAP code (#32849) 2024-12-15 11:59:18 +00:00
TheFox0x7 33e8e82c4b
Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
Rowan Bohde a03fdd9566
Avoid MacOS keychain dialog in integration tests (#32813)
Mac's git installation ships with a system wide config that configures
the credential helper `osxkeychain`, which will prompt the user with a
dialog.

```
$ git config list --system 
credential.helper=osxkeychain
```
By setting the environment variable
[`GIT_CONFIG_NOSYSTEM=true`](https://git-scm.com/docs/git-config#ENVIRONMENT),
Git will not load the system wide config, preventing the dialog from
populating.

Closes #26717
2024-12-12 21:10:47 +00:00
wxiaoguang 22bf2ca6ba
Make API "compare" accept commit IDs (#32801) 2024-12-12 08:10:09 +00:00
Kemal Zebari 01b1896bf5
Implement update branch API (#32433)
Resolves #22526.

Builds upon #23061.

---------

Co-authored-by: sillyguodong <33891828+sillyguodong@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-12-12 05:02:35 +00:00
Sebastian T. T. 874b8484aa
Add standard-compliant route to serve outdated R packages (#32783)
The R package repository currently does not have support for older
versions of packages which should be stored in a separate /Archive
router. This PR remedies that by adding a new path router.

I am a member of a group that loves using Gitea and this bug has been
annoying us for a long time. Hope it can be merged in time for Gitea
1.23.0.

Any feedback much appreciated.

Fixes #32782
2024-12-11 16:20:04 +00:00
wxiaoguang 4814f43af7
Fix repo home file list (#32788)
1. use grid instead of table, completely drop "ui table" from that list
2. move some "commit sign" related styles into a new file by the way (no
change) because I need to figure out where `#repo-files-table` is used.
3. move legacy "branch/tag selector" related code into repo-legacy.ts,
now there are 13 `import $` files left.
2024-12-11 23:54:42 +08:00
Blender Defender 18061af490
Rearrange Clone Panel (#31142)
Rearrange the clone panel to use less horizontal space.
The following changes have been made to achieve this:
- Moved everything into the dropdown menu
- Moved the HTTPS/SSH Switch to a separate line
- Moved the "Clone in VS Code"-Button up and added a divider
- Named the dropdown button "Code", added appropriate icon

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-12-11 21:54:30 +08:00
Lunny Xiao fbe6d9dc6b
Use batch database operations instead of one by one to optimze api pulls (#32680)
Resolve #31492

The response time for the Pull Requests API has improved significantly,
dropping from over `2000ms` to about `350ms` on my local machine. It's
about `6` times faster.

A key area for further optimization lies in batch-fetching data for
`apiPullRequest.ChangedFiles, apiPullRequest.Additions, and
apiPullRequest.Deletions`.

Tests `TestAPIViewPulls` does exist and new tests added.

- This PR also fixes some bugs in `GetDiff` functions.
- This PR also fixes data inconsistent in test data. For a pull request,
the head branch's reference should be equal to the reference in
`pull/xxx/head`.
2024-12-11 05:15:06 +08:00
Lunny Xiao b32f0cdfa0
Make wiki pages visit fast (#32732) 2024-12-06 07:39:50 +00:00
Illya Marchenko 936665bf85
Issue time estimate, meaningful time tracking (#23113)
Redesign the time tracker side bar, and add "time estimate" support (in "1d 2m" format)

Closes #23112

---------

Co-authored-by: stuzer05 <stuzer05@gmail.com>
Co-authored-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-12-05 13:07:53 +00:00
KN4CK3R 0c3c041c88
Add Arch package registry (#32692)
Close #25037
Close #31037

This PR adds a Arch package registry usable with pacman.

![grafik](https://github.com/user-attachments/assets/81cdb0c2-02f9-4733-bee2-e48af6b45224)

Rewrite of #25396 and #31037. You can follow [this
tutorial](https://wiki.archlinux.org/title/Creating_packages) to build a
package for testing.

Docs PR: https://gitea.com/gitea/docs/pulls/111

Co-authored-by: [d1nch8g@ion.lc](mailto:d1nch8g@ion.lc)
Co-authored-by: @ExplodingDragon

---------

Co-authored-by: dancheg97 <dancheg97@fmnx.su>
Co-authored-by: dragon <ExplodingFKL@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-12-04 23:09:07 +00:00
Lunny Xiao 17053e953f
Fix delete branch perm checking (#32654) 2024-12-04 11:59:48 +08:00
KN4CK3R 136408307c
Add Swift login endpoint (#32693)
Fix #32683

This PR adds the login endpoint and fixes the documentation links.
2024-12-03 16:24:16 +00:00
wxiaoguang def13ece7c
Allow to disable the password-based login (sign-in) form (#32687)
Usually enterprise/organization users would like to only allow OAuth2
login.

This PR adds a new config option to disable the password-based login
form. It is a simple and clear approach and won't block the future
login-system refactoring works.

Fix a TODO in #24821

Replace  #21851

Close #7633 , close #13606
2024-12-02 02:03:15 +08:00
Lunny Xiao 79d593a9be
Split mail sender sub package from mailer service package (#32618)
Move all mail sender related codes into a sub package of
services/mailer. Just move, no code change.
Then we just have dependencies on go-mail package in the new sub
package. We can use other package to replace it because it's
unmaintainable. ref #18664
2024-11-30 01:15:41 +00:00
Zettat123 fd3aa5bedb
Fix a bug in actions artifact test (#32672)
This bug exists in `TestActionsArtifactDownload`.


a1f56f83bf/tests/integration/api_actions_artifact_test.go (L123-L134)

We assert that `listResp.Count` is `2`, so `artifactIdx` could be `0` or `1`.

a1f56f83bf/tests/integration/api_actions_artifact_test.go (L144-L147)

Then we assert that the length of `downloadResp.Value` is `1`. If
`artifactIdx` is `1` at this point, the assertion on Line 147 will throw
an `index out of range` error.
2024-11-29 20:32:10 +00:00
Rowan Bohde 16a7d343d7
Validate OAuth Redirect URIs (#32643)
This fixes a TODO in the code to validate the RedirectURIs when adding
or editing an OAuth application in user settings.

This also includes a refactor of the user settings tests to only create
the DB once per top-level test to avoid reloading fixtures.
2024-11-28 10:50:27 +08:00
Lunny Xiao f62f68cbdd
Move team related functions to service layer (#32537)
There are still some functions under `models` after last big refactor
about `models`. This change will move all team related functions to
service layer with no code change.
2024-11-27 21:12:26 +00:00
6543 846f618716
Add priority to protected branch (#32286)
## Solves

Currently for rules to re-order them you have to alter the creation
date. so you basicly have to delete and recreate them in the right
order. This is more than just inconvinient ...

## Solution

Add a new col for prioritization

## Demo WebUI Video

https://github.com/user-attachments/assets/92182a31-9705-4ac5-b6e3-9bb74108cbd1


---
*Sponsored by Kithara Software GmbH*
2024-11-27 05:41:06 +01:00
wxiaoguang b6ce2d6dc9
Refactor markup render system (#32645)
This PR mainly removes some global variables, moves some code and
renames some functions to make code clearer.

This PR also removes a testing-only option ForceHardLineBreak during
refactoring since the behavior is clear now.
2024-11-26 16:46:02 +00:00
wxiaoguang 9ed768adc4
Improve oauth2 scope token handling (#32633) 2024-11-26 02:03:02 +00:00
Kerwin Bryant 25cacaf0aa
Fixed Issue of Review Menu Shown Behind (#32631)
Fixed #31144 

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-26 09:24:56 +08:00
Lunny Xiao 703be6bf30
Add github compatible tarball download API endpoints (#32572)
Fix #29654 
Fix #32481
2024-11-25 19:35:49 +00:00
wxiaoguang 5d57c287fb
Fix sqlite3 test (#32622) 2024-11-25 06:10:11 +00:00
wxiaoguang 633785a5f3
Refactor markup render system (#32612)
This PR removes (almost) all path tricks, and introduces "renderhelper"
package.

Now we can clearly see the rendering behaviors for comment/file/wiki,
more details are in "renderhelper" tests.

Fix #31411 , fix #18592, fix #25632 and maybe more problems. (ps: fix
#32608 by the way)
2024-11-24 16:18:57 +08:00
Lunny Xiao fe49cb0243
Fix get reviewers' bug (#32415)
This PR rewrites `GetReviewer` function and move it to service layer.

Reviewers should not be watchers, so that this PR removed all watchers
from reviewers. When the repository is under an organization, the pull
request unit read permission will be checked to resolve the bug of
#32394

Fix #32394
2024-11-22 10:44:48 -05:00
wxiaoguang c4e27cb27b
Refactor markup render system (#32589)
This PR mainly moves some code and introduces `RenderContext.WithXxx`
functions
2024-11-22 05:48:09 +00:00
Marcell Mars a3881ffa3d
Enhancing Gitea OAuth2 Provider with Granular Scopes for Resource Access (#32573)
Resolve #31609

This PR was initiated following my personal research to find the
lightest possible Single Sign-On solution for self-hosted setups. The
existing solutions often seemed too enterprise-oriented, involving many
moving parts and services, demanding significant resources while
promising planetary-scale capabilities. Others were adequate in
supporting basic OAuth2 flows but lacked proper user management
features, such as a change password UI.

Gitea hits the sweet spot for me, provided it supports more granular
access permissions for resources under users who accept the OAuth2
application.

This PR aims to introduce granularity in handling user resources as
nonintrusively and simply as possible. It allows third parties to inform
users about their intent to not ask for the full access and instead
request a specific, reduced scope. If the provided scopes are **only**
the typical ones for OIDC/OAuth2—`openid`, `profile`, `email`, and
`groups`—everything remains unchanged (currently full access to user's
resources). Additionally, this PR supports processing scopes already
introduced with [personal
tokens](https://docs.gitea.com/development/oauth2-provider#scopes) (e.g.
`read:user`, `write:issue`, `read:group`, `write:repository`...)

Personal tokens define scopes around specific resources: user info,
repositories, issues, packages, organizations, notifications,
miscellaneous, admin, and activitypub, with access delineated by read
and/or write permissions.

The initial case I wanted to address was to have Gitea act as an OAuth2
Identity Provider. To achieve that, with this PR, I would only add
`openid public-only` to provide access token to the third party to
authenticate the Gitea's user but no further access to the API and users
resources.

Another example: if a third party wanted to interact solely with Issues,
it would need to add `read:user` (for authorization) and
`read:issue`/`write:issue` to manage Issues.

My approach is based on my understanding of how scopes can be utilized,
supported by examples like [Sample Use Cases: Scopes and
Claims](https://auth0.com/docs/get-started/apis/scopes/sample-use-cases-scopes-and-claims)
on auth0.com.

I renamed `CheckOAuthAccessToken` to `GetOAuthAccessTokenScopeAndUserID`
so now it returns AccessTokenScope and user's ID. In the case of
additional scopes in `userIDFromToken` the default `all` would be
reduced to whatever was asked via those scopes. The main difference is
the opportunity to reduce the permissions from `all`, as is currently
the case, to what is provided by the additional scopes described above.

Screenshots:

![Screenshot_20241121_121405](https://github.com/user-attachments/assets/29deaed7-4333-4b02-8898-b822e6f2463e)

![Screenshot_20241121_120211](https://github.com/user-attachments/assets/7a4a4ef7-409c-4116-9d5f-2fe00eb37167)

![Screenshot_20241121_120119](https://github.com/user-attachments/assets/aa52c1a2-212d-4e64-bcdf-7122cee49eb6)

![Screenshot_20241121_120018](https://github.com/user-attachments/assets/9eac318c-e381-4ea9-9e2c-3a3f60319e47)
---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-22 12:06:41 +08:00
Lunny Xiao a175f9805c
Add line-through for deleted branch on pull request view page (#32500)
Base branch deleted

![图片](https://github.com/user-attachments/assets/bc45aa33-d514-47c8-885a-de9732f2f3d5)

branch deleted comment

![图片](https://github.com/user-attachments/assets/83729bbb-2ee8-4bd3-b6f1-780d2daad3d4)

Head branch deleted

![图片](https://github.com/user-attachments/assets/90120b22-34a9-4387-aae9-5c882e8d421a)
2024-11-21 17:04:18 +00:00
Rowan Bohde 9ac74a1a40
disable gravatar in test (#32529)
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
2024-11-21 04:30:48 +00:00
Lunny Xiao 696fbe6036
Refactor push mirror find and add check for updating push mirror (#32539)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-18 05:59:04 +00:00
Lunny Xiao 4f879a00df
Refactor find forks and fix possible bugs that weak permissions check (#32528)
- Move models/GetForks to services/FindForks
- Add doer as a parameter of FindForks to check permissions
- Slight performance optimization for get forks API with batch loading
of repository units
- Add tests for forking repository to organizations

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-18 03:06:25 +00:00
Lunny Xiao c3dedcffa7
Fix basic auth with webauthn (#32531) 2024-11-16 17:52:16 +00:00
wxiaoguang ecbb03dc6d
Improve testing and try to fix MySQL hanging (#32515)
By some CI fine tunes (`run tests`), SQLite & MSSQL could complete
in about 12~13 minutes (before > 14), MySQL could complete in 18 minutes
(before: about 23 or even > 30)

Major changes:

1. use tmpfs for MySQL storage
1. run `make test-mysql` instead of `make integration-test-coverage`
because the code coverage is not really used at the moment.
1. refactor testlogger to make it more reliable and be able to report
stuck stacktrace
1. do not requeue failed items when a queue is being flushed (failed
items would keep failing and make flush uncompleted)
1. reduce the file sizes for testing
1. use math ChaCha20 random data instead of crypot/rand (for testing
purpose only)
1. no need to `DeleteRepository` in `TestLinguist`
1. other related refactoring to make code easier to maintain
2024-11-15 23:45:07 +08:00
Rowan Bohde 68731c07c5
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
wxiaoguang 3f9c3e7bc3
Refactor render system (#32492)
There were too many patches to the Render system, it's really difficult
to make further improvements.

This PR clears the legacy problems and fix TODOs.

1. Rename `RenderContext.Type` to `RenderContext.MarkupType` to clarify
its usage.
2. Use `ContentMode` to replace `meta["mode"]` and `IsWiki`, to clarify
the rendering behaviors.
3. Use "wiki" mode instead of "mode=gfm + wiki=true"
4. Merge `renderByType` and `renderByFile`
5. Add more comments

----

The problem of "mode=document": in many cases it is not set, so many
non-comment places use comment's hard line break incorrectly
2024-11-14 05:02:11 +00:00
wxiaoguang 0aedb03996
Fix LFS route mock, realm, middleware names (#32488)
1. move "internal-lfs" route mock to "common-lfs"
2. fine tune tests
3. fix "realm" strings, according to RFC:
https://datatracker.ietf.org/doc/html/rfc2617:
    * realm       = "realm" "=" realm-value
    * realm-value = quoted-string
4. clarify some names of the middlewares, rename `ignXxx` to `optXxx` to
match `reqXxx`, and rename ambiguous `requireSignIn` to `reqGitSignIn`
2024-11-13 16:58:09 +08:00
wxiaoguang 160ccb5ee2
Fix test fixtures for user2/lfs.git (#32477) 2024-11-12 05:41:22 +00:00
wxiaoguang 580e21dd2e
Refactor LFS SSH and internal routers (#32473)
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:

* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)

Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
2024-11-12 02:38:22 +00:00
Calvin K 18aeca5320
Add reviewers selection to new pull request (#32403)
Users could add reviewers when creating new PRs.

---------

Co-authored-by: splitt3r <splitt3r@users.noreply.github.com>
Co-authored-by: Sebastian Sauer <sauer.sebastian@gmail.com>
Co-authored-by: bb-ben <70356237+bboerben@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-09 04:48:31 +00:00
Zettat123 35bcd667b2
Fix broken releases when re-pushing tags (#32435)
Fix #32427
2024-11-08 10:53:06 +08:00
Kemal Zebari 7adc4717ec
Include file extension checks in attachment API (#32151)
From testing, I found that issue posters and users with repository write
access are able to edit attachment names in a way that circumvents the
instance-level file extension restrictions using the edit attachment
APIs. This snapshot adds checks for these endpoints.
2024-11-06 21:34:32 +00:00
Bruno Sofiato f64fbd9b74
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-11-06 20:51:20 +00:00
Lunny Xiao 24b83ff63e
Fix milestone deadline and date related problems (#32339)
Use zero instead of 9999-12-31 for deadline
Fix #32291

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-11-05 07:46:40 +00:00
wxiaoguang 61c35590c7
Refactor RepoRefByType (#32413)
1. clarify the "filepath" could(should) contain "{ref}"
2. remove unclear RepoRefLegacy and RepoRefAny, use RepoRefUnknown to guess
3. by the way, avoid using AppURL
2024-11-05 06:35:54 +00:00
wxiaoguang ec2d1593c2
Refactor tests to prevent from unnecessary preparations (#32398) 2024-11-01 23:18:29 +08:00
Kyle D. 66971e591e
Add artifacts test fixture (#30300)
Closes https://github.com/go-gitea/gitea/issues/30296

- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
2024-11-01 10:29:54 +08:00
wxiaoguang a920fcfd91
Fix db engine (#32351)
Fix #32349
2024-10-27 18:48:07 -04:00
Tim de2ad2e1b1
Make admins adhere to branch protection rules (#32248)
This introduces a new flag `BlockAdminMergeOverride` on the branch
protection rules that prevents admins/repo owners from bypassing branch
protection rules and merging without approvals or failing status checks.

Fixes #17131

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-10-23 12:39:43 +08:00