Commit Graph

89 Commits

Author SHA1 Message Date
zeripath 33a8eec33e
Retry rename on lock induced failures (#16435)
* Retry rename on lock induced failures

Due to external locking on Windows it is possible for an
os.Rename to fail if the files or directories are being
used elsewhere.

This PR simply suggests retrying the rename again similar
to how we handle the os.Remove problems.

Fix #16427

Signed-off-by: Andrew Thornton <art27@cantab.net>

* resolve CI fail

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 11:46:07 -04:00
luzpaz e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
KN4CK3R 440039c0cc
Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
silverwind 1e6fa57acb
Use single shared random string generation function (#15741)
* Use single shared random string generation function

- Replace 3 functions that do the same with 1 shared one
- Use crypto/rand over math/rand for a stronger RNG
- Output only alphanumerical for URL compatibilty

Fixes: #15536

* use const string method

* Update modules/avatar/avatar.go

Co-authored-by: a1012112796 <1012112796@qq.com>

Co-authored-by: a1012112796 <1012112796@qq.com>
2021-05-10 07:45:17 +01:00
KN4CK3R c03e488e14
Add LFS Migration and Mirror (#14726)
* Implemented LFS client.

* Implemented scanning for pointer files.

* Implemented downloading of lfs files.

* Moved model-dependent code into services.

* Removed models dependency. Added TryReadPointerFromBuffer.

* Migrated code from service to module.

* Centralised storage creation.

* Removed dependency from models.

* Moved ContentStore into modules.

* Share structs between server and client.

* Moved method to services.

* Implemented lfs download on clone.

* Implemented LFS sync on clone and mirror update.

* Added form fields.

* Updated templates.

* Fixed condition.

* Use alternate endpoint.

* Added missing methods.

* Fixed typo and make linter happy.

* Detached pointer parser from gogit dependency.

* Fixed TestGetLFSRange test.

* Added context to support cancellation.

* Use ReadFull to probably read more data.

* Removed duplicated code from models.

* Moved scan implementation into pointer_scanner_nogogit.

* Changed method name.

* Added comments.

* Added more/specific log/error messages.

* Embedded lfs.Pointer into models.LFSMetaObject.

* Moved code from models to module.

* Moved code from models to module.

* Moved code from models to module.

* Reduced pointer usage.

* Embedded type.

* Use promoted fields.

* Fixed unexpected eof.

* Added unit tests.

* Implemented migration of local file paths.

* Show an error on invalid LFS endpoints.

* Hide settings if not used.

* Added LFS info to mirror struct.

* Fixed comment.

* Check LFS endpoint.

* Manage LFS settings from mirror page.

* Fixed selector.

* Adjusted selector.

* Added more tests.

* Added local filesystem migration test.

* Fixed typo.

* Reset settings.

* Added special windows path handling.

* Added unit test for HTTPClient.

* Added unit test for BasicTransferAdapter.

* Moved into util package.

* Test if LFS endpoint is allowed.

* Added support for git://

* Just use a static placeholder as the displayed url may be invalid.

* Reverted to original code.

* Added "Advanced Settings".

* Updated wording.

* Added discovery info link.

* Implemented suggestion.

* Fixed missing format parameter.

* Added Pointer.IsValid().

* Always remove model on error.

* Added suggestions.

* Use channel instead of array.

* Update routers/repo/migrate.go

* fmt

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-08 18:25:57 -04:00
6543 0d1444751f
[API] Add pagination to ListBranches (#14524)
* make PaginateUserSlice generic -> PaginateSlice

* Add pagination to ListBranches

* add skip, limit to Repository.GetBranches()

* Move routers/api/v1/utils/utils PaginateSlice -> modules/util/paginate.go

* repo_module.GetBranches paginate

* fix & rename & more logging

* better description

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: a1012112796 <1012112796@qq.com>
2021-02-03 20:06:13 +01:00
6543 a19447aed1
migrate from com.* to alternatives (#14103)
* remove github.com/unknwon/com from models

* dont use "com.ToStr()"

* replace "com.ToStr" with "fmt.Sprint" where its easy to do

* more refactor

* fix test

* just "proxy" Copy func for now

* as per @lunny
2020-12-25 11:59:32 +02:00
Lunny Xiao acd5e5a868
Add StatDir and replace com.StatDir (#14099)
* Add StatDir and replace com.StatDir

* a nit

* Remove wrong file

Co-authored-by: 6543 <6543@obermui.de>
2020-12-22 07:40:57 +08:00
zeripath 742e21aeba
Handle and propagate errors when checking if paths are Dirs, Files or Exist (#13186)
* Ensure errors from IsDir propagate

* Handle errors when checking IsFile

* Handle and propagate errors from IsExist

* Update modules/templates/static.go

* Update modules/templates/static.go

* Return after ctx.ServerError

* Apply suggestions from code review

* Fix tests

The previous merge managed to break repo_form.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-27 21:42:08 -05:00
silverwind 6d93a3ab18
Issue and Pulls lists rework (#13594)
* Issue and Pulls lists rework

Reorganized and restyled the issue and pull request lists.

* color and layout tweaks

* use new issue list on dashboard as well

* move pagination into template

* misc tweaks

* fix label hover

* fix milestone list

* fix discrepancies between issue and milestone list, add new 'merge' helper

* fmt

* simplify merge helper

* remove whitespace

* fix startIndex

* further simplify dict merging

* rename helper to 'mergeinto' for clarity

* allow bottom-row to wrap

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-25 13:20:40 +02:00
kolaente 64133126cd
Update golangci-lint to version 1.31.0 (#13102)
This PR updates golangci-lint to the latest version 1.31.0.

The upgrade introduced a new check for which I've fixed or disabled most cases.

Signed-off-by: kolaente <k@knt.li>
2020-10-11 21:27:20 +01:00
zeripath 4c6ac08182
Completely quote AppPath and CustomConf paths (#12955)
* Completely quote AppPath and CustomConf paths

Properly handle spaces in AppPath and CustomConf within hooks and
authorized_keys. Unfortunately here we don't seem to be able to get away
with using go-shellquote as it appears that Windows doesn't play too
well with singlequote quoting - therefore we will avoid singlequote
quoting unless we absolutely cannot get away without it, e.g. \n or !.

Fix #10813

Signed-off-by: Andrew Thornton <art27@cantab.net>

* missing change

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix Test_CmdKeys

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-09-28 21:16:52 -04:00
zeripath 74bd9691c6
Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.

Fix #12339

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
2020-08-11 21:05:34 +01:00
zeripath c58bc4bf80
Prevent timer leaks in Workerpool and others (#11333)
There is a potential memory leak in `Workerpool` due to the intricacies of
`time.Timer` stopping.

Whenever a `time.Timer` is `Stop`ped its channel must be cleared using a
`select` if the result of the `Stop()` is `false`.

Unfortunately in `Workerpool` these were checked the wrong way round.

However, there were a few other places that were not being checked.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-05-08 16:46:05 +01:00
6543 694f44660f
inform participants on UI too (#10473)
* inform participants on UI too

* ajust test

* refactor getParticipantIDsByIssue
2020-02-28 05:16:41 -03:00
Lunny Xiao a67c06ce90 Sanitize credentials in mirror form (#9975) 2020-01-25 10:57:42 +00:00
guillep2k 7b97e04555 Convert EOL to UNIX-style to render MD properly (#8925)
* Convert EOL to UNIX-style to render MD properly

* Update modules/markup/markdown/markdown.go

Co-Authored-By: zeripath <art27@cantab.net>

* Fix lint optimization

* Check for empty content before conversion

* Update modules/util/util.go

Co-Authored-By: zeripath <art27@cantab.net>

* Improved checks and tests

* Add paragraph render test

* Improve speed even more, improve tests

* Small improvement by @gary-kim

* Fix test for DOS

* More improvements

* Restart CI
2019-11-13 03:27:11 +01:00
zeripath ee1d64ddd1
Stop using git count-objects and use raw directory size for repository (#8848)
* Migrate from git count-objects to a raw directory size
* As per @guillep2k ignore unusual files
2019-11-10 21:33:47 +00:00
blueworrybear 8c909820a9 Enable Uploading/Removing Attachments When Editing an Issue/Comment (#8426) 2019-10-15 13:19:32 +01:00
Lunny Xiao 85202d4784
Display ui time with customize time location (#7792)
* display ui time with customize time location

* fix lint

* rename UILocation to DefaultUILocation

* move time related functions to modules/timeutil

* fix tests

* fix tests

* fix build

* fix swagger
2019-08-15 22:46:21 +08:00
zeripath 69d81b6569
Handle insecure and ports in go get (#7041)
* Handle insecure and ports in go get

* Fix IsExternalURL for non-standard ports
2019-05-27 22:08:38 +01:00
zeripath 704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
mrsdizzie ca46385637 Clean up various use of escape/unescape functions for URL generation (#6334)
* Use PathUnescape instead of QueryUnescape when working with branch names

Currently branch names with a '+' fail in certain situations because
QueryUnescape replaces the + character with a blank space.

Using PathUnescape should be better since it is defined as:

// PathUnescape is identical to QueryUnescape except that it does not
// unescape '+' to ' ' (space).

Fixes #6333

* Change error to match new function name

* Add new util function PathEscapeSegments

This function simply runs PathEscape on each segment of a path without
touching the forward slash itself. We want to use this instead of
PathEscape/QueryEscape in most cases because a forward slash is a valid name for a
branch etc... and we don't want that escaped in a URL.

Putting this in new file url.go and also moving a couple similar
functions into that file as well.

* Use EscapePathSegments where appropriate

Replace various uses of EscapePath/EscapeQuery with new
EscapePathSegments. Also remove uncessary uses of various
escape/unescape functions when the text had already been escaped or was
not escaped.

* Reformat comment to make drone build happy

* Remove no longer used url library

* Requested code changes
2019-03-18 10:00:23 -04:00
zeripath dbab98c8e0 Remove util.RemoveAll - should have been removed since go 1.7 (#6299) 2019-03-10 23:56:36 +02:00
Lanre Adelowo e1fcd6b742 Disallow empty titles (#5785)
* add util method and tests

* make sure the title of an issue cannot be empty

* wiki title cannot be empty

* pull request title cannot be empty

* update to make use of the new util methof
2019-01-21 13:45:32 +02:00
Jonas Franz 801843b011 Fix open redirect vulnerability on login screen (#4312)
* Fix open redirect vulnerability on login screen

Signed-off-by: Jonas Franz <info@jonasfranz.software>

* Reorder imports

Signed-off-by: Jonas Franz <info@jonasfranz.software>

* Replace www. from Domain too

Signed-off-by: Jonas Franz <info@jonasfranz.software>
2018-06-26 22:45:18 +08:00
Antoine GIRARD 35c3510335 Fix #4081 Check for leading / in base before removing it (#4082) 2018-05-30 21:23:43 +08:00
Jonas Franz 2139c152cb Fix relative markdown links with anchors (#4058)
* Replace '%28' with '#'
Add test case

Signed-off-by: Jonas Franz <info@jonasfranz.software>

* Use ResolveReference instead of strings.Replace

Signed-off-by: Jonas Franz <info@jonasfranz.software>
2018-05-28 23:51:42 -04:00
Magnus Lindvall cdb9478774 LDAP Public SSH Keys synchronization (#1844)
* Add LDAP Key Synchronization feature

Signed-off-by: Magnus Lindvall <magnus@dnmgns.com>

* Add migration: add login source id column for public_key table

* Only update keys if needed

* Add function to only list pubkey synchronized from ldap

* Only list pub ssh keys synchronized from ldap. Do not sort strings as ExistsInSlice does it.

* Only get keys belonging to current login source id

* Set default login source id to 0

* Some minor cleanup. Add integration tests (updete dep testify)
2018-05-24 07:59:02 +03:00
Lunny Xiao 769ab1e424 fix gpg expired bug when time is zero (#3584) 2018-02-26 12:25:45 +02:00
Ethan Koenig 7b104f0cd0 Populate URL field of API commits (#3546)
* Populate URL field of API commits

* fix orgmode_test
2018-02-20 20:50:42 +08:00
Lunny Xiao f94c1b3943
Improvements for supporting UI Location (#3146)
* improvements for supporting UI Location

* improved the comment
2017-12-14 07:45:31 +08:00
Lunny Xiao f2e20c81b6 Refactor struct's time to remove unnecessary memory usage (#3142)
* refactor struct's time to remove unnecessary memory usage

* use AsTimePtr simple code

* fix tests

* fix time compare

* fix template on gpg

* use AddDuration instead of Add
2017-12-11 06:37:04 +02:00
Ethan Koenig 3c1b1ca78e Fix error message sanitiziation (#3082) 2017-12-04 03:48:03 +02:00
Ethan Koenig 5866eb2321 Code/repo search (#2582)
Indexed search of repository contents (for default branch only)
2017-10-27 09:10:54 +03:00
Lunny Xiao 6eeadb2082 Hide unactive on explore users and some refactors (#2741)
* hide unactive on explore users and some refactors

* fix test for removed Organizations

* fix test for removed Organizations

* fix imports

* fix logic bug

* refactor the toConds

* Rename TestOrganizations to TestSearchUsers and add tests for users

* fix other tests

* fix other tests

* fix watchers tests

* fix comments and remove unused code
2017-10-24 20:36:19 +03:00
Lunny Xiao 1739e84ac0 improve protected branch to add whitelist support (#2451)
* improve protected branch to add whitelist support

* fix lint

* fix style check

* fix tests

* fix description on UI and import

* fix test

* bug fixed

* fix tests and languages

* move isSliceInt64Eq to util pkg; improve function names & typo
2017-09-14 16:16:22 +08:00
Bo-Yi Wu 08f7fded3c refactor: remove workaround after the golang 1.7 release. (#1349)
* refactor: remove workaround after the golang 1.7 release.

* remove unused import.

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>

* refactor: Add remove function.

* refactor: rename

* refactor: update build flag
2017-03-29 10:05:23 +08:00
Ethan Koenig 833f8b94c2 Search bar for issues/pulls (#530) 2017-01-25 10:43:02 +08:00