* Graceful Shutdown for windows and others
Restructures modules/graceful, adding shutdown for windows, removing and
replacing the old minwinsvc code.
Creates a new waitGroup - terminate which allows for goroutines to
finish up after the shutdown of the servers.
Shutdown and terminate hooks are added for goroutines.
* Remove unused functions - these can be added in a different PR
* Add startup timeout functionality
* Document STARTUP_TIMEOUT
In investigating #7947 it has become clear that the storage component of go-git repositories needs closing.
This PR adds this Close function and adds the Close functions as necessary.
In TransferOwnership the ctx.Repo.GitRepo is closed if it is open to help prevent the risk of multiple open files.
Fixes#7947
* Only attempt to kill parent once
* Apply suggestions from code review
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Add waitgroup for running servers
Add an option to protected branches to add writing deploy keys to the whitelist for pushing.
Please note this is technically a breaking change: previously if the owner of a repository was on the whitelist then any writing deploy key was effectively on the whitelist. This option will now need to be set if that is desired.
Closes#8472
Details:
* Allow Protected Branches to Whitelist Deploy Keys
* Add migration
* Ensure that IsDeployKey is set to false on the http pushes
* add not null default false
* Prevent deadlock in indexer initialisation during graceful restart
* Move from gracehttp to our own service to add graceful ssh
* Add timeout for start of indexers and make hammer time configurable
* Fix issue with re-initialization in indexer during tests
* move the code to detect use of closed to graceful
* Handle logs gracefully - add a pid suffix just before restart
* Move to using a cond and a holder for indexers
* use time.Since
* Add some comments and attribution
* update modules.txt
* Use zero to disable timeout
* Move RestartProcess to its own file
* Add cleanup routine
Add password complexity checks. The default settings require a lowercase, uppercase, number and a special character within passwords.
Co-Authored-By: T-M-A <maxim.tkachenko@gmail.com>
Co-Authored-By: Lanre Adelowo <adelowomailbox@gmail.com>
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-Authored-By: Lauris BH <lauris@nix.lv>
Regression in 356854fc5f, where
a log.Fatal call was removed. log.Fatal calls os.Exit(1) as intended,
but without it the fail() function returns normally and execution
continues past the fatal error, causing a panic.
This is visible as a go panic log and stack trace returned to the SSH
client, which is not only ugly, it leaks server and build system
information.
Fix by removing the stray return statement so that the fail() function
always calls os.Exit(1).
Fixes: https://github.com/go-gitea/gitea/issues/7457
Signed-off-by: Allen Wild <allenwild93@gmail.com>
* Fix#6946 by checking PullRequest ID on pushing
* Ensure we have the owner name, the pr attributes and the the issue
* Fix TestSearchRepo by waiting till indexing is done
* Update integrations/repo_search_test.go
* changes as per @mrsdizzie
* missing comma
* Spelling mistake
* Fix full pushing environment
* add CLI commands to manage LDAP authentication source
* delete Gogs copyright
* remove unused return value of func parseLoginSource
* fix comment
Co-Authored-By: ngourdon <31291059+ngourdon@users.noreply.github.com>
* remove config flag already present in global flags
* remove config flag from ldap commands in docs
* remove config flag handling
* Move hook functionality internally
* Internalise serv logic
* Remove old internal paths
* finally remove the gitlogger
* Disallow push on archived repositories
* fix lint error
* Update modules/private/key.go
* Update routers/private/hook.go
* Update routers/private/hook.go
* Update routers/private/hook.go
* Updated routers/private/serv.go
* Fix LFS Locks over SSH
* rev-list needs to be run by the hook process
* fixup
* Improve git test
* Ensure that the lfs files are created with a different prefix
* Reduce the replication in git_test.go
* slight refactor
* Remove unnecessary "/"
* Restore ensureAnonymousClone
* Restore ensureAnonymousClone
* Run rev-list on server side
* Try passing in the alternative directories instead
* Mark test as skipped
* Improve git test
* Ensure that the lfs files are created with a different prefix
* Reduce the replication in git_test.go
* Remove unnecessary "/"
When replicating to gitea from a remote system which makes use of
git refs to store extra data (for example, gerrit), pushing a lot
of refs to gitea can cause problems due to the extra processing
that the pre and post receive hooks perform. But it's still
useful for gitea to be able to serve those refs. This change
skips unecessary processing of refs other than branches or tags.
We don't need to check any ref that isn't a branch for branch
protection (protection will never be enabled). So in the
pre-receive hook, we wrap that check in a test for whether the
ref is a branch.
We also don't need to add information to the activity stream about
pushes to non-standard refs, so we skip that step in the
post-receive hook for refs which are not branches or tags.
For some concrete examples, gerrit maintains a ref for every
patchset of every change in the form refs/changes/XX/YYYY/Z.
Many systems use refs/notes to store additonal data about commits.
This change allows these and other schemes to be used without
affecting gitea.
* Add username flag in create-user command
* Fix the error message
* Change name to username on flag of create-user command
* Include create-user flags of both versions
* delete deprecated example of create-user command
When running the dump command, include the specific configuration file
provided as part of the backup. Currently we only zip up custom/ and this
misses the config file if it is somewhere else, like /etc/gitea/app.ini
Fixes#6034
* Panic don't fatal on create new logger
Fixes#5854
Signed-off-by: Andrew Thornton <art27@cantab.net>
* partial broken
* Update the logging infrastrcture
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reset the skip levels for Fatal and Error
Signed-off-by: Andrew Thornton <art27@cantab.net>
* broken ncsa
* More log.Error fixes
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove nal
* set log-levels to lowercase
* Make console_test test all levels
* switch to lowercased levels
* OK now working
* Fix vetting issues
* Fix lint
* Fix tests
* change default logging to match current gitea
* Improve log testing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* reset error skip levels to 0
* Update documentation and access logger configuration
* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE
* Fix broken level caching
* Refactor the router log
* Add Router logger
* Add colorizing options
* Adjust router colors
* Only create logger if they will be used
* update app.ini.sample
* rename Attribute ColorAttribute
* Change from white to green for function
* Set fatal/error levels
* Restore initial trace logger
* Fix Trace arguments in modules/auth/auth.go
* Properly handle XORMLogger
* Improve admin/config page
* fix fmt
* Add auto-compression of old logs
* Update error log levels
* Remove the unnecessary skip argument from Error, Fatal and Critical
* Add stacktrace support
* Fix tests
* Remove x/sync from vendors?
* Add stderr option to console logger
* Use filepath.ToSlash to protect against Windows in tests
* Remove prefixed underscores from names in colors.go
* Remove not implemented database logger
This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.
* Ensure that log paths are relative to ROOT_PATH
* use path.Join
* rename jsonConfig to logConfig
* Rename "config" to "jsonConfig" to make it clearer
* Requested changes
* Requested changes: XormLogger
* Try to color the windows terminal
If successful default to colorizing the console logs
* fixup
* Colorize initially too
* update vendor
* Colorize logs on default and remove if this is not a colorizing logger
* Fix documentation
* fix test
* Use go-isatty to detect if on windows we are on msys or cygwin
* Fix spelling mistake
* Add missing vendors
* More changes
* Rationalise the ANSI writer protection
* Adjust colors on advice from @0x5c
* Make Flags a comma separated list
* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING
* Ensure matching is done on the non-colored message - to simpify EXPRESSION
* Use PathUnescape instead of QueryUnescape when working with branch names
Currently branch names with a '+' fail in certain situations because
QueryUnescape replaces the + character with a blank space.
Using PathUnescape should be better since it is defined as:
// PathUnescape is identical to QueryUnescape except that it does not
// unescape '+' to ' ' (space).
Fixes#6333
* Change error to match new function name
* Add new util function PathEscapeSegments
This function simply runs PathEscape on each segment of a path without
touching the forward slash itself. We want to use this instead of
PathEscape/QueryEscape in most cases because a forward slash is a valid name for a
branch etc... and we don't want that escaped in a URL.
Putting this in new file url.go and also moving a couple similar
functions into that file as well.
* Use EscapePathSegments where appropriate
Replace various uses of EscapePath/EscapeQuery with new
EscapePathSegments. Also remove uncessary uses of various
escape/unescape functions when the text had already been escaped or was
not escaped.
* Reformat comment to make drone build happy
* Remove no longer used url library
* Requested code changes
1. A key can either be an ssh user key or a deploy key. It cannot be both.
2. If a key is a user key - it can only be associated with one user.
3. If a key is a deploy key - it can be used in multiple repositories and the permissions it has on those repositories can be different.
4. If a repository is deleted, its deploy keys must be deleted too.
We currently don't enforce any of this and multiple repositories access with different permissions doesn't work at all. This PR enforces the following constraints:
- [x] You should not be able to add the same user key as another user
- [x] You should not be able to add a ssh user key which is being used as a deploy key
- [x] You should not be able to add a ssh deploy key which is being used as a user key
- [x] If you add an ssh deploy key to another repository you should be able to use it in different modes without losing the ability to use it in the other mode.
- [x] If you delete a repository you must delete all its deploy keys.
Fix#1357
The gitea prerecieve and postrecieve hooks and the gitea PushUpdate function require that the PusherID and PusherName are real users. Previously, these environment variables were not being set when using a deploy key - the main result being that pushing to empty repositories meant that is_empty status was not changed.
I've also added an integration test to ensure that the is_empty status is updated on pushing with a deploy key.
There is a slight issue in that the deploy key is now considered a proxy for the owner - we don't have a way of separating out the deploy key from the owner at present. This can be fixed in another PR.
Fix#3795
Signed-off-by: Andrew Thornton art27@cantab.net
By default, if `setting.NewContext()` prints out any warning logs, these are printed to the stdout breaking `git receive-pack` etc. meaning that even if there is a warning because of a minor problem in your app.ini but gitea starts despite this - you **CANNOT** push or pull over SSH.
This PR disables the console logger whilst in `serv.go`
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add util method and tests
* make sure the title of an issue cannot be empty
* wiki title cannot be empty
* pull request title cannot be empty
* update to make use of the new util methof
* add migration and basic UI for changing a user's theme
* update user themem
* use right text on button
* load theme based on users' selection
* load theme based on users' selection in pwa too
* update sample config
* delete older theme loading
* implement AfterLoad to set users' theme properly
* set up default theme when creating a user. This uses the installation wide theme
* use flash messages for error
* set default theme when creating a user from the cli
* fix @lunny review
* Fix the Let's Encrypt handler by listening on a valid address
Also handle errors in the HTTP server go routine, return a fatal error
when something goes wrong.
Thanks to @gbl08ma for finding the actual bug
Here is an example of the error handling:
2018/12/11 14:23:07 [....io/gitea/cmd/web.go:87 func1()] [E] Failed to
start the Let's Encrypt handler on port 30: listen tcp 0.0.0.0:30: bind:
permission denied
Closes#5280
* Fix a typo
* don't force a password change for the admin user
* don't totally dicard -must-change-password flag if creating the first (admin) user via the cli. Use flag if present but make sure to default to not forcing a password update
* add support for an admin to force a user to change his/her password from thee cli
* use BoolFlag instead
* default to true
* simplify by removing unnneccessary if/else
* Add repo-sync-releases admin command
Will help recovering corrupted database, see #3247
* Load repos in chunks of 10, exit with error if unable to get a list, scan private repos, fix typo
* Fix debug output about num releases
* Introduce RepositoryListDefaultPageSize constant, set to 64
Use it from the new admin command
* Use RepositoryListDefaultPageSize in more places
* Document RepositoryListDefaultPageSize
* Add an option to allow redirect of http port 80 to https.
This is an "opt in" option (default is to not redirect). It will only redirect
if protocol is https and the new REDIRECT_PORT_80 option is set to true.
The Port to redirect in previous commit was hardcoded to 80, now it can be
specified in the app.ini, defaulting to 80. The boolean option to turn
redirection on has been changed to REDIRECT_OTHER_PORT to be logically
consistent with the new port option.
Signed-off-by: Mike Fellows <mike.fellows@shaw.ca>
* Set session and indexers' data files rel to AppDataPath
The setting AppDataPath is now relative to the working directory.
The session svc's PROVIDER_CONFIG now defaults to AppDataPath/data/sessions.
The issue indexer's IssuePath now defaults to AppDataPath/indexers/issues.bleves.
* fix bug
* add units to team
* fix lint
* finish team setting backend
* finished permission controll on routes
* fix import blank line
* add unit check on ssh/http pull and push and fix test failed
* fix fixtures data
* remove unused code
* move all database operations from hook command to web command and instead of internal routes
* bug fixed
* adjust the import path sequences
* remove unused return value on hookSetup
* add internal routes for ssh hook comands
* fix lint
* add comment on why package named private not internal but the route name is internal
* add comment above package private why package named private not internal but the route name is internal
* remove exp time on internal access
* move routes from /internal to /api/internal
* add comment and defer on UpdatePublicKeyUpdated
* Add correct git branch name validation
* Change git refname validation error constant name
* Implement URL validation based on GoLang url.Parse method
* Backward compatibility with older Go compiler
* Add git reference name validation unit tests
* Remove unused variable in unit test
* Implement URL validation based on GoLang url.Parse method
* Backward compatibility with older Go compiler
* Add url validation unit tests
* Allow ENABLE_OPENID_SIGNUP to depend on DISABLE_REGISTRATION
Omit the configuration variable (the default) to be dependent.
Fixes#1363
* Move OpenID settings under Service object
* Show OpenID SignUp and SignIn status in admin panel / configuration
Also renames `forgot_password` translation key to
`forgot_password_title` and `forget_password` to
`forgot_password`
Includes entry in CHANGELOG about the breaking change
(and some markdown fixes in there)
* move all push update to git hook post-receive and protected branch check to git hook pre-receive
* add SSH_ORIGINAL_COMMAND check back
* remove all unused codes
* fix the import
* initial stuff for oauth2 login, fails on:
* login button on the signIn page to start the OAuth2 flow and a callback for each provider
Only GitHub is implemented for now
* show login button only when the OAuth2 consumer is configured (and activated)
* create macaron group for oauth2 urls
* prevent net/http in modules (other then oauth2)
* use a new data sessions oauth2 folder for storing the oauth2 session data
* add missing 2FA when this is enabled on the user
* add password option for OAuth2 user , for use with git over http and login to the GUI
* add tip for registering a GitHub OAuth application
* at startup of Gitea register all configured providers and also on adding/deleting of new providers
* custom handling of errors in oauth2 request init + show better tip
* add ExternalLoginUser model and migration script to add it to database
* link a external account to an existing account (still need to handle wrong login and signup) and remove if user is removed
* remove the linked external account from the user his settings
* if user is unknown we allow him to register a new account or link it to some existing account
* sign up with button on signin page (als change OAuth2Provider structure so we can store basic stuff about providers)
* from gorilla/sessions docs:
"Important Note: If you aren't using gorilla/mux, you need to wrap your handlers with context.ClearHandler as or else you will leak memory!"
(we're using gorilla/sessions for storing oauth2 sessions)
* use updated goth lib that now supports getting the OAuth2 user if the AccessToken is still valid instead of re-authenticating (prevent flooding the OAuth2 provider)
* Protected branches system
* Moved default branch to branches section (`:org/:reponame/settings/branches`).
* Initial support Protected Branch.
- Admin does not restrict
- Owner not to limit
- To write permission restrictions
* reformat tmpl
* finished the UI and add/delete protected branch response
* remove unused comment
* indent all the template files and remove ru translations since we use crowdin
* fix the push bug
* Markdown rendering overhaul
Cleaned up and squashed commits into single one.
Signed-off-by: Andrew Boyarshin <boyarshinand@gmail.com>
* Fix markdown API, add markdown module and API tests, improve code coverage
Signed-off-by: Andrew Boyarshin <boyarshinand@gmail.com>
This change corrects a few logging issues:
* Standardized formatting errors with '%v'.
* Standardized failure warning word usage.
* Corrected an instance of using the standard log library when
the gitea log library should be used instead.
* Moved attachaments POST url from /issues/attachments to /attachments
* Implemented attachment upload on release page
* Implemented downloading attachments on the release page
* Added zip and gzip files to default allowed attachments
* Implemented uploading attachments on edit release
* Renamed UploadIssueAttachment to UploadAttachment
* Use relative URLs
* Notifications - Mark as read/unread
* Feature of pinning a notification
* On view issue, do not mark as read a pinned notification
* Tab on user profile to show starred repos
* Make golint happy and use transactions on StarRepo function
* x -> sess
* Use sess.Close() instead of sess.Rollback()
* Add copyright
* Fix lint
* Import github.com/git-lfs/lfs-test-server as lfs module base
Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198
Removed:
Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go
.dockerignore .gitignore README.md
* Remove config, add JWT support from github.com/mgit-at/lfs-test-server
Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83
* Add LFS settings
* Add LFS meta object model
* Add LFS routes and initialization
* Import github.com/dgrijalva/jwt-go into vendor/
* Adapt LFS module: handlers, routing, meta store
* Move LFS routes to /user/repo/info/lfs/*
* Add request header checks to LFS BatchHandler / PostHandler
* Implement LFS basic authentication
* Rework JWT secret generation / load
* Implement LFS SSH token authentication with JWT
Specification: https://github.com/github/git-lfs/tree/master/docs/api
* Integrate LFS settings into install process
* Remove LFS objects when repository is deleted
Only removes objects from content store when deleted repo is the only
referencing repository
* Make LFS module stateless
Fixes bug where LFS would not work after installation without
restarting Gitea
* Change 500 'Internal Server Error' to 400 'Bad Request'
* Change sql query to xorm call
* Remove unneeded type from LFS module
* Change internal imports to code.gitea.io/gitea/
* Add Gitea authors copyright
* Change basic auth realm to "gitea-lfs"
* Add unique indexes to LFS model
* Use xorm count function in LFS check on repository delete
* Return io.ReadCloser from content store and close after usage
* Add LFS info to runWeb()
* Export LFS content store base path
* LFS file download from UI
* Work around git-lfs client issue with unauthenticated requests
Returning a dummy Authorization header for unauthenticated requests
lets git-lfs client skip asking for auth credentials
See: https://github.com/github/git-lfs/issues/1088
* Fix unauthenticated UI downloads from public repositories
* Authentication check order, Finish LFS file view logic
* Ignore LFS hooks if installed for current OS user
Fixes Gitea UI actions for repositories tracking LFS files.
Checks for minimum needed git version by parsing the semantic version
string.
* Hide LFS metafile diff from commit view, marking as binary
* Show LFS notice if file in commit view is tracked
* Add notbefore/nbf JWT claim
* Correct lint suggestions - comments for structs and functions
- Add comments to LFS model
- Function comment for GetRandomBytesAsBase64
- LFS server function comments and lint variable suggestion
* Move secret generation code out of conditional
Ensures no LFS code may run with an empty secret
* Do not hand out JWT tokens if LFS server support is disabled
* Moved conf assets into options folder
* Dropped old bindata
* Started to integrate options bindata and accessors
* Do not enforce a builtin app.ini
* Replaced bindata calls with options
* Dropped bindata task from makefile, it's the generate task now
* Always embedd app.ini to provide sane config defaults
* Use sane defaults for the configuration
* Defined default value for SSH_KEYGEN_PATH
* Dropped "NEVER EVER MODIFY THIS FILE" header from app.ini
* Fixed new paths in latest test additions
* Drop bindata with make clean task
* Set more proper default values
Export Pusher name as GITEA_PUSHER_NAME env variable
Export also GITEA_UUID, but keep the uuid env variable for backward compatibility
export pusher name ENV variable #316
change env variable prefix to GITEA_
Signed-off-by: Antonio Facciolo <afdev82@gmail.com>
Export also GITEA_UUID #316
Keep uuid env variable for backward compatibility
* Get rid of the bin folder within the build process
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Dropped latest make task, it is unused
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Added tidb tag to drone config
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Dropped the cert build tag
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Dropped useless minwinsvc build tag
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Dropped the useless build tags from drone config
Signed-off-by: Thomas Boerger <thomas@webhippie.de>
* Dropped unused codekit config
* Integrated dynamic and static bindata for public
* Ignore public bindata
* Add a general generate make task
* Integrated flexible public assets into web command
* Updated vendoring, added all missiong govendor deps
* Made the linter happy with the bindata and dynamic code
* Moved public bindata definition to modules directory
* Ignoring the new bindata path now
* Updated to the new public modules import path
* Updated public bindata command and drop the new prefix
- Download filename was wrong for files other than images. Example: It was `download` instead of `file.pdf`
- PDF was downloading instead of showing on browser
* Can use a predefined set of labels
* Change UI
* Fix HTML file indentation
* Avoid reading file from other directory (security issue)
* Apply a better fix
* Remove not used variable
* Merge upstream/develop
* Do modifications
* Raname
* remove binding + rename variable
* Add support for federated avatars
Fixes#3105
Removes avatar fetching duplication code
Adds an "Enable Federated Avatar" checkbox in user settings
(defaults to unchecked)
Moves avatar settings all in the same form, making
local and remote avatars mutually exclusive
Renames UploadAvatarForm to AvatarForm
as it's not anymore only for uploading
* Run gofmt on all modified files
* Move Avatar form in its own page
* Add go-libravatar dependency to vendor/ dir
Hopefully helps with accepting the contribution.
See also #3214
* Revert "Add go-libravatar dependency to vendor/ dir"
This reverts commit a8cb93ae640bbb90f7d25012fc257bda9fae9b82.
* Make federated avatar setting a global configuration
Removes the per-user setting
* Move avatar handling back to base tool, disable federated avatar in offline mode
* Format, handle error
* Properly set fallback host
* Use unsupported github.com mirror for importing go-libravatar
* Remove comment showing life exists outside of github.com
... pity, but contribution would not be accepted otherwise
* Use Combo for Get and Post methods over /avatar
* FEDERATED_AVATAR -> ENABLE_FEDERATED_AVATAR
* Fix persistance of federated avatar lookup checkbox at install time
* Federated Avatars -> Enable Federated Avatars
* Use len(string) == 0 instead of string == ""
* Move import line where it belong
See
https://github.com/Unknwon/go-code-convention/blob/master/en-US/import_packages.md
Pity the import url is still the unofficial one, but oh well...
* Save a line (and waste much more expensive time)
* Remove redundant parens
* Remove an empty line
* Remove empty lines
* Reorder lines to make diff smaller
* Remove another newline
Unknwon review got me start a fight against newlines
* Move DISABLE_GRAVATAR and ENABLE_FEDERATED_AVATAR after OFFLINE_MODE
On re-reading the diff I figured what Unknwon meant here:
https://github.com/gogits/gogs/pull/3320/files#r73741106
* Remove newlines that weren't there before my intervention
Remove the possibility of using email as user name when user actually push
through combination of email and password with HTTP.
Also refactor update action function to replcae tons of arguments with
single PushUpdateOptions struct.
And define the user who pushes code as pusher, therefore variable names shouldn't
be confusing any more.
zeripath