Commit Graph

461 Commits

Author SHA1 Message Date
wxiaoguang c248f010ad
Refactor cache and disable go-chi cache (#30417)
use built-in cache package to wrap external go-chi cache package
2024-04-13 08:38:44 +00:00
wxiaoguang f9fdac9809
Limit the max line length when parsing git grep output (#30418) 2024-04-12 03:36:34 +00:00
wxiaoguang 0db554fa63
Refactor commit signature parser (#30228)
To make it more flexible and support SSH signature.

The existing tests are not changed, there are also tests covering
`parseTagRef` which also calls `parsePayloadSignature` now. Add some new
tests to `Test_parseTagData`
2024-04-01 20:23:17 +00:00
KN4CK3R b6a3cd4b8d
Include encoding in signature payload (#30174)
Fixes #30119

Include the encoding in the signature payload.

before

![grafik](https://github.com/go-gitea/gitea/assets/1666336/01ab94a3-8af5-4d6f-be73-a10b65a15421)

after

![grafik](https://github.com/go-gitea/gitea/assets/1666336/3a37d438-c70d-4d69-b178-d170e74aa683)
2024-03-29 21:55:10 +00:00
wxiaoguang 8e79aed573
Fix git grep search limit, add test (#30071)
Fix #30069
2024-03-25 21:25:22 +08:00
wxiaoguang 4734d43e14
Support repo code search without setting up an indexer (#29998)
By using git's ability, end users (especially small instance users) do
not need to enable the indexer, they could also benefit from the code
searching feature.

Fix #29996


![image](https://github.com/go-gitea/gitea/assets/2114189/11b7e458-88a4-480d-b4d7-72ee59406dd1)


![image](https://github.com/go-gitea/gitea/assets/2114189/0fe777d5-c95c-4288-a818-0427680805b6)

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 17:05:00 +01:00
wxiaoguang 226231ea27
Fix some pending problems (#29985)
These changes are quite independent and trivial, so I don't want to open
too many PRs.

* https://github.com/go-gitea/gitea/pull/29882#discussion_r1529607091
    * the `f.Close` should be called properly
* the error message could be more meaningful
(https://github.com/go-gitea/gitea/pull/29882#pullrequestreview-1942557935)
*
https://github.com/go-gitea/gitea/pull/29859#pullrequestreview-1942324716
    * the new translation strings don't take arguments
* https://github.com/go-gitea/gitea/pull/28710#discussion_r1443778807
    * stale for long time
*  #28140 
    * a form was forgotten to be changed to work with backend code
2024-03-22 19:17:30 +08:00
wxiaoguang 21151474e3
Fix loadOneBranch panic (#29938)
Try to fix #29936

Far from ideal, but still better than panic.
2024-03-20 15:38:22 +00:00
coldWater 0e183d81fc
Fix missing error check of bufio.Scanner (#29882)
maybe more
2024-03-19 02:20:36 +00:00
Lunny Xiao e84e5db6de
Lazy load object format with command line and don't do it in OpenRepository (#29712)
Most time, when invoking `git.OpenRepository`, `objectFormat` will not
be used, so it's a waste to invoke commandline to get the object format.
This PR make it a lazy operation, only invoke that when necessary.
2024-03-12 04:21:27 +00:00
pengqiseven 7f856d5d74
remove repetitive words (#29695)
Signed-off-by: pengqiseven <912170095@qq.com>
2024-03-11 09:24:23 +00:00
Lunny Xiao 25b842df26
Move get/set default branch from git package to gitrepo package to hide repopath (#29126) 2024-03-08 15:30:10 +08:00
wxiaoguang 5cddab4f74
Make wiki default branch name changable (#29603)
Fix #29000
Fix #28685
Fix #18568

Related: #27497

And by the way fix #24036, add a Cancel button there (one line)
2024-03-06 14:26:32 +08:00
charles 423372d84a
Add a check for when the command is canceled by the program on Window… (#29538)
Close #29509

Windows, unlike Linux, does not have signal-specified exit codes.
Therefore, we should add a Windows-specific check for Windows. If we
don't do this, the logs will always show a failed status, even though
the command actually works correctly.

If you check the Go source code in exec_windows.go, you will see that it
always returns exit code 1.

![image](https://github.com/go-gitea/gitea/assets/30816317/9dfd7c70-9995-47d9-9641-db793f58770c)

The exit code 1 does not exclusively signify a SIGNAL KILL; it can
indicate any issue that occurs when a program fails.
2024-03-02 13:38:34 +00:00
KN4CK3R f79c9e817a
Use `crypto/sha256` (#29386)
Go 1.21 improved the performance of `crypto/sha256`. It's now similar to
`minio/sha256-simd`, so we should just use the standard libs.

https://go.dev/doc/go1.21#crypto/sha256
https://go-review.googlesource.com/c/go/+/408795
https://github.com/multiformats/go-multihash/pull/173
2024-02-25 13:32:13 +00:00
KN4CK3R ad0a34b492
Add `io.Closer` guidelines (#29387)
Co-authored-by: Yarden Shoham <git@yardenshoham.com>
2024-02-25 13:05:23 +00:00
wxiaoguang 1ef87773b1
Refactor modules/git global variables (#29376)
Move some global variables into a struct to improve maintainability
2024-02-25 05:35:47 +00:00
KN4CK3R 4197e28100
Refactor git attributes (#29356) 2024-02-24 18:46:49 +00:00
KN4CK3R 553d46e6f6
Do not double close reader (#29354)
Fixes #29346

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-02-24 11:45:59 +00:00
KN4CK3R 2a278b996f
Add support for `linguist-detectable` and `linguist-documentation` (#29267)
Add support for `linguist-detectable` and `linguist-documentation`
Add tests for the attributes


https://github.com/github-linguist/linguist/blob/master/docs/overrides.md#detectable

https://github.com/github-linguist/linguist/blob/master/docs/overrides.md#documentation
2024-02-23 17:24:27 +00:00
KN4CK3R d6811baf88
Discard unread data of `git cat-file` (#29297)
Fixes #29101
Related #29298

Discard all read data to prevent misinterpreting existing data. Some
discard calls were missing in error cases.

---------

Co-authored-by: yp05327 <576951401@qq.com>
2024-02-22 03:48:19 +00:00
KN4CK3R f74c869221
Prevent double use of `git cat-file` session. (#29298)
Fixes the reason why #29101 is hard to replicate.
Related #29297

Create a repo with a file with minimum size 4097 bytes (I use 10000) and
execute the following code:
```go
gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>)
assert.NoError(t, err)

commit, err := gitRepo.GetCommit(<sha>)
assert.NoError(t, err)

entry, err := commit.GetTreeEntryByPath(<file>)
assert.NoError(t, err)

b := entry.Blob()

// Create a reader
r, err := b.DataAsync()
assert.NoError(t, err)
defer r.Close()

// Create a second reader
r2, err := b.DataAsync()
assert.NoError(t, err) // Should be no error but is ErrNotExist
defer r2.Close()
```

The problem is the check in `CatFileBatch`:

79217ea63c/modules/git/repo_base_nogogit.go (L81-L87)
`Buffered() > 0` is used to check if there is a "operation" in progress
at the moment. This is a problem because we can't control the internal
buffer in the `bufio.Reader`. The code above demonstrates a sequence
which initiates an operation for which the code thinks there is no
active processing. The second call to `DataAsync()` therefore reuses the
existing instances instead of creating a new batch reader.
2024-02-21 19:54:17 +01:00
techknowlogick a346a8c852
bump to use go 1.22 (#29119) 2024-02-14 18:19:57 +00:00
wxiaoguang d0183dfa49
Refactor git version functions and check compatibility (#29155)
Introduce a new function checkGitVersionCompatibility, when the git
version can't be used by Gitea, tell the end users to downgrade or
upgrade. The refactored functions are related to make the code easier to
test.

And simplify the comments for "safe.directory"

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-02-14 17:18:30 +00:00
Chris Copeland 47b5965862
Add merge style `fast-forward-only` (#28954)
With this option, it is possible to require a linear commit history with
the following benefits over the next best option `Rebase+fast-forward`:
The original commits continue existing, with the original signatures
continuing to stay valid instead of being rewritten, there is no merge
commit, and reverting commits becomes easier.

Closes #24906
2024-02-12 23:37:23 +01:00
wxiaoguang a24e1da7e9
Refactor parseSignatureFromCommitLine (#29054)
Replace #28849. Thanks to @yp05327 for the looking into the problem.
Fix #28840

The old behavior of newSignatureFromCommitline is not right. The new
parseSignatureFromCommitLine:

1. never fails
2. only accept one format (if there is any other, it could be easily added)

And add some tests.
2024-02-09 11:02:53 +08:00
Lunny Xiao 5f82ead13c
Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.

## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:

Firstly, there are temporary repositories which do not change:

```go
git.OpenRepository(ctx, diskPath)
```

Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:

```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```

Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.

## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-01-27 21:09:51 +01:00
John Olheiser 76bbad4e74
Check for sha256 support to use --object-format flag (#28928)
This should fix https://github.com/go-gitea/gitea/issues/28927

Technically older versions of Git would support this flag as well, but
per https://github.com/go-gitea/gitea/pull/28466 that's the version
where using it (object-format=sha256) left "experimental" state.

`sha1` is (currently) the default, so older clients should be unaffected
in either case.

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2024-01-25 13:04:50 -05:00
wxiaoguang 82acf22d9c
Update go dependencies and fix go-git (#28893)
More details are in the comment of repo_base_gogit.go

And ref: https://github.com/go-git/go-git/issues/1006
2024-01-23 05:40:00 +00:00
Adam Majer d68a613ba8
Add support for sha256 repositories (#23894)
Currently only SHA1 repositories are supported by Gitea. This adds
support for alternate SHA256 with the additional aim of easier support
for additional hash types in the future.

Fixes: #13794
Limited by: https://github.com/go-git/go-git/issues/899
Depend on: #28138

<img width="776" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/5448c9a7-608e-4341-a149-5dd0069c9447">

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2024-01-19 17:05:02 +01:00
Brecht Van Lommel 1167d523c4
Fix archive creating LFS hooks and breaking pull requests (#28848)
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.

Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.

But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.

According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.

Resolves #19810, #21148
2024-01-19 05:49:18 +00:00
Lunny Xiao c8ba17c73f
Remove duplicated checkinit on git module (#28824)
`checkInit` has been invoked in `InitSimple`. So it's unnecessary to
invoke it twice in `InitFull`.
2024-01-17 09:56:00 +00:00
Lunny Xiao 930e38d010
Use refname:strip-2 instead of refname:short when syncing tags (#28797)
Fix #28694 

Generally, `refname:short` should be equal to `refname:lstrip=2` except
`core.warnAmbiguousRefs is used to select the strict abbreviation mode.`

ref:
https://git-scm.com/docs/git-for-each-ref#Documentation/git-for-each-ref.txt-refname
2024-01-16 08:13:07 +01:00
Mihir Joshi 669bbbaf24
Integration Test for Commit Search containing Square Brackets (#28751)
Integration test for #28744 

Change keywords commit search flag from `-F` to `--fixed-strings` for
readability
2024-01-11 11:04:45 +08:00
Mihir Joshi 839cd26b1a
Add -F to commit search to treat keywords as strings (#28744)
Fixes #28269

The [default
behavior](https://git-scm.com/docs/git-log#Documentation/git-log.txt---basic-regexp)
of --grep in git log is to interpret the keyword as a regular
expression. This causes the search to fail in the cases where the search
keyword contains a `[`, since `[` is a special character used in grep.

If we want our keywords to be interpreted as 'strings', we should use
[-F
flag](https://git-scm.com/docs/git-log#Documentation/git-log.txt---basic-regexp).
2024-01-10 11:28:20 +00:00
Lunny Xiao e7cb8da2a8
Always enable caches (#28527)
Nowadays, cache will be used on almost everywhere of Gitea and it cannot
be disabled, otherwise some features will become unaviable.

Then I think we can just remove the option for cache enable. That means
cache cannot be disabled.
But of course, we can still use cache configuration to set how should
Gitea use the cache.
2023-12-19 09:29:05 +00:00
Lunny Xiao 4eb2a29910
Improve ObjectFormat interface (#28496)
The 4 functions are duplicated, especially as interface methods. I think
we just need to keep `MustID` the only one and remove other 3.

```
MustID(b []byte) ObjectID
MustIDFromString(s string) ObjectID
NewID(b []byte) (ObjectID, error)
NewIDFromString(s string) (ObjectID, error)
```

Introduced the new interfrace method `ComputeHash` which will replace
the interface `HasherInterface`. Now we don't need to keep two
interfaces.

Reintroduced `git.NewIDFromString` and `git.MustIDFromString`. The new
function will detect the hash length to decide which objectformat of it.
If it's 40, then it's SHA1. If it's 64, then it's SHA256. This will be
right if the commitID is a full one. So the parameter should be always a
full commit id.

@AdamMajer Please review.
2023-12-19 07:20:47 +00:00
wxiaoguang 11f0519ad8
Update go dependencies (#28518)
Update golang.org/x/crypto for CVE-2023-48795 and update other packages.
`go-git` is not updated because it needs time to figure out why some
tests fail.
2023-12-19 09:18:42 +08:00
wxiaoguang 20929edc99
Add option to disable ambiguous unicode characters detection (#28454)
* Close #24483
* Close #28123
* Close #23682
* Close #23149

(maybe more)
2023-12-17 14:38:54 +00:00
Lunny Xiao 408a484224
Adjust object format interface (#28469)
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
2023-12-17 11:56:08 +00:00
wxiaoguang 9947af639c
Only use SHA256 feature when git >= 2.42 (#28466)
And fix some comments
2023-12-14 16:51:05 +08:00
Adam Majer cbf923e87b
Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
Adam Majer d9c19899cd
Make gogit Repository.GetBranchNames consistent (#28348) 2023-12-07 12:08:17 -05:00
darrinsmart 38a93a0665
Convert git commit summary to valid UTF8. (#28356)
The summary string ends up in the database, and (at least) MySQL &
PostgreSQL require valid UTF8 strings.

Fixes #28178

Co-authored-by: Darrin Smart <darrin@filmlight.ltd.uk>
2023-12-05 14:34:24 +08:00
KN4CK3R 14354e4f8e
Read `previous` info from git blame (#28306)
Fixes #28280

Reads the `previous` info from the `git blame` output instead of
calculating it afterwards.
2023-12-01 01:26:52 +00:00
silverwind ce83609ff6
Upgrade to golangci-lint@v1.55.0 (#27756)
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
Nanguan Lin dc04044716
Replace assert.Fail with assert.FailNow (#27578)
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
2023-10-11 11:02:24 +00:00
Eng Zer Jun 13d5d2e711
Remove redundant `len` check around loop (#27464)
This pull request is a minor code cleanup.

From the Go specification (https://go.dev/ref/spec#For_range):

> "1. For a nil slice, the number of iterations is 0."
> "3. If the map is nil, the number of iterations is 0."

`len` returns 0 if the slice or map is nil
(https://pkg.go.dev/builtin#len). Therefore, checking `len(v) > 0`
before a loop is unnecessary.

---

At the time of writing this pull request, there wasn't a lint rule that
catches these issues. The closest I could find is
https://staticcheck.dev/docs/checks/#S103

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-10-06 14:49:37 +08:00
wxiaoguang dfa4e5857f
Fix git 2.11 error when checking IsEmpty (again) (#27399)
Follow  #27393

Sorry that I made a mistake in #27393. The `errbuf` is not empty when
the err is `exit status 129`.
2023-10-02 15:59:39 +00:00
wxiaoguang caef9f9503
Fix git 2.11 error when checking IsEmpty (#27393)
Fix #27389
2023-10-02 22:05:21 +08:00