gitea/modules/markup/html_test.go

704 lines
28 KiB
Go
Raw Normal View History

// Copyright 2017 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package markup_test
import (
"context"
"io"
"os"
"strings"
"testing"
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
2023-06-20 23:50:26 -06:00
"code.gitea.io/gitea/models/unittest"
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
"code.gitea.io/gitea/modules/emoji"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/markdown"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
"github.com/stretchr/testify/assert"
)
var localMetas = map[string]string{
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
2019-08-14 02:04:55 -06:00
"user": "gogits",
"repo": "gogs",
"repoPath": "../../tests/gitea-repositories-meta/user13/repo11.git/",
}
func TestMain(m *testing.M) {
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
2023-06-20 23:50:26 -06:00
unittest.InitSettings()
if err := git.InitSimple(context.Background()); err != nil {
log.Fatal("git init failed, err: %v", err)
}
os.Exit(m.Run())
}
func TestRender_Commits(t *testing.T) {
setting.AppURL = markup.TestAppURL
test := func(input, expected string) {
buffer, err := markup.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
RelativePath: ".md",
Links: markup.Links{
AbsolutePrefix: true,
Base: markup.TestRepoURL,
},
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
}
sha := "65f1bf27bc3bf70f64657658635e66094edbcb4d"
repo := "http://localhost:3000/gogits/gogs"
commit := util.URLJoin(repo, "commit", sha)
tree := util.URLJoin(repo, "tree", sha, "src")
file := util.URLJoin(repo, "commit", sha, "example.txt")
fileWithExtra := file + ":"
fileWithHash := file + "#L2"
fileWithHasExtra := file + "#L2:"
commitCompare := util.URLJoin(repo, "compare", sha+"..."+sha)
commitCompareWithHash := commitCompare + "#L2"
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
2019-08-14 02:04:55 -06:00
test(sha, `<p><a href="`+commit+`" rel="nofollow"><code>65f1bf27bc</code></a></p>`)
test(sha[:7], `<p><a href="`+commit[:len(commit)-(40-7)]+`" rel="nofollow"><code>65f1bf2</code></a></p>`)
test(sha[:39], `<p><a href="`+commit[:len(commit)-(40-39)]+`" rel="nofollow"><code>65f1bf27bc</code></a></p>`)
test(commit, `<p><a href="`+commit+`" rel="nofollow"><code>65f1bf27bc</code></a></p>`)
test(tree, `<p><a href="`+tree+`" rel="nofollow"><code>65f1bf27bc/src</code></a></p>`)
test(file, `<p><a href="`+file+`" rel="nofollow"><code>65f1bf27bc/example.txt</code></a></p>`)
test(fileWithExtra, `<p><a href="`+file+`" rel="nofollow"><code>65f1bf27bc/example.txt</code></a>:</p>`)
test(fileWithHash, `<p><a href="`+fileWithHash+`" rel="nofollow"><code>65f1bf27bc/example.txt (L2)</code></a></p>`)
test(fileWithHasExtra, `<p><a href="`+fileWithHash+`" rel="nofollow"><code>65f1bf27bc/example.txt (L2)</code></a>:</p>`)
test(commitCompare, `<p><a href="`+commitCompare+`" rel="nofollow"><code>65f1bf27bc...65f1bf27bc</code></a></p>`)
test(commitCompareWithHash, `<p><a href="`+commitCompareWithHash+`" rel="nofollow"><code>65f1bf27bc...65f1bf27bc (L2)</code></a></p>`)
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
2019-08-14 02:04:55 -06:00
test("commit "+sha, `<p>commit <a href="`+commit+`" rel="nofollow"><code>65f1bf27bc</code></a></p>`)
test("/home/gitea/"+sha, "<p>/home/gitea/"+sha+"</p>")
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
2019-08-14 02:04:55 -06:00
test("deadbeef", `<p>deadbeef</p>`)
test("d27ace93", `<p>d27ace93</p>`)
test(sha[:14]+".x", `<p>`+sha[:14]+`.x</p>`)
expected14 := `<a href="` + commit[:len(commit)-(40-14)] + `" rel="nofollow"><code>` + sha[:10] + `</code></a>`
test(sha[:14]+".", `<p>`+expected14+`.</p>`)
test(sha[:14]+",", `<p>`+expected14+`,</p>`)
test("["+sha[:14]+"]", `<p>[`+expected14+`]</p>`)
}
func TestRender_CrossReferences(t *testing.T) {
setting.AppURL = markup.TestAppURL
test := func(input, expected string) {
buffer, err := markup.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
RelativePath: "a.md",
Links: markup.Links{
AbsolutePrefix: true,
Base: setting.AppSubURL,
},
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
}
test(
"test-owner/test-repo#12345",
`<p><a href="`+util.URLJoin(markup.TestAppURL, "test-owner", "test-repo", "issues", "12345")+`" class="ref-issue" rel="nofollow">test-owner/test-repo#12345</a></p>`)
test(
"go-gitea/gitea#12345",
`<p><a href="`+util.URLJoin(markup.TestAppURL, "go-gitea", "gitea", "issues", "12345")+`" class="ref-issue" rel="nofollow">go-gitea/gitea#12345</a></p>`)
test(
"/home/gitea/go-gitea/gitea#12345",
`<p>/home/gitea/go-gitea/gitea#12345</p>`)
test(
util.URLJoin(markup.TestAppURL, "gogitea", "gitea", "issues", "12345"),
`<p><a href="`+util.URLJoin(markup.TestAppURL, "gogitea", "gitea", "issues", "12345")+`" class="ref-issue" rel="nofollow">gogitea/gitea#12345</a></p>`)
test(
util.URLJoin(markup.TestAppURL, "go-gitea", "gitea", "issues", "12345"),
`<p><a href="`+util.URLJoin(markup.TestAppURL, "go-gitea", "gitea", "issues", "12345")+`" class="ref-issue" rel="nofollow">go-gitea/gitea#12345</a></p>`)
test(
util.URLJoin(markup.TestAppURL, "gogitea", "some-repo-name", "issues", "12345"),
`<p><a href="`+util.URLJoin(markup.TestAppURL, "gogitea", "some-repo-name", "issues", "12345")+`" class="ref-issue" rel="nofollow">gogitea/some-repo-name#12345</a></p>`)
inputURL := "https://host/a/b/commit/0123456789012345678901234567890123456789/foo.txt?a=b#L2-L3"
test(
inputURL,
`<p><a href="`+inputURL+`" rel="nofollow"><code>0123456789/foo.txt (L2-L3)</code></a></p>`)
}
func TestMisc_IsSameDomain(t *testing.T) {
setting.AppURL = markup.TestAppURL
sha := "b6dd6210eaebc915fd5be5579c58cce4da2e2579"
commit := util.URLJoin(markup.TestRepoURL, "commit", sha)
assert.True(t, markup.IsSameDomain(commit))
assert.False(t, markup.IsSameDomain("http://google.com/ncr"))
assert.False(t, markup.IsSameDomain("favicon.ico"))
}
func TestRender_links(t *testing.T) {
setting.AppURL = markup.TestAppURL
test := func(input, expected string) {
buffer, err := markup.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
RelativePath: "a.md",
Links: markup.Links{
Base: markup.TestRepoURL,
},
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
}
oldCustomURLSchemes := setting.Markdown.CustomURLSchemes
markup.ResetDefaultSanitizerForTesting()
defer func() {
setting.Markdown.CustomURLSchemes = oldCustomURLSchemes
markup.ResetDefaultSanitizerForTesting()
markup.CustomLinkURLSchemes(oldCustomURLSchemes)
}()
setting.Markdown.CustomURLSchemes = []string{"ftp", "magnet"}
markup.CustomLinkURLSchemes(setting.Markdown.CustomURLSchemes)
// Text that should be turned into URL
test(
"https://www.example.com",
`<p><a href="https://www.example.com" rel="nofollow">https://www.example.com</a></p>`)
test(
"http://www.example.com",
`<p><a href="http://www.example.com" rel="nofollow">http://www.example.com</a></p>`)
test(
"https://example.com",
`<p><a href="https://example.com" rel="nofollow">https://example.com</a></p>`)
test(
"http://example.com",
`<p><a href="http://example.com" rel="nofollow">http://example.com</a></p>`)
test(
"http://foo.com/blah_blah",
`<p><a href="http://foo.com/blah_blah" rel="nofollow">http://foo.com/blah_blah</a></p>`)
test(
"http://foo.com/blah_blah/",
`<p><a href="http://foo.com/blah_blah/" rel="nofollow">http://foo.com/blah_blah/</a></p>`)
test(
"http://www.example.com/wpstyle/?p=364",
`<p><a href="http://www.example.com/wpstyle/?p=364" rel="nofollow">http://www.example.com/wpstyle/?p=364</a></p>`)
test(
"https://www.example.com/foo/?bar=baz&inga=42&quux",
`<p><a href="https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux" rel="nofollow">https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux</a></p>`)
test(
"http://142.42.1.1/",
`<p><a href="http://142.42.1.1/" rel="nofollow">http://142.42.1.1/</a></p>`)
test(
"https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd",
`<p><a href="https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd" rel="nofollow">https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd</a></p>`)
test(
"https://en.wikipedia.org/wiki/URL_(disambiguation)",
`<p><a href="https://en.wikipedia.org/wiki/URL_(disambiguation)" rel="nofollow">https://en.wikipedia.org/wiki/URL_(disambiguation)</a></p>`)
test(
"https://foo_bar.example.com/",
`<p><a href="https://foo_bar.example.com/" rel="nofollow">https://foo_bar.example.com/</a></p>`)
test(
"https://stackoverflow.com/questions/2896191/what-is-go-used-fore",
`<p><a href="https://stackoverflow.com/questions/2896191/what-is-go-used-fore" rel="nofollow">https://stackoverflow.com/questions/2896191/what-is-go-used-fore</a></p>`)
test(
"https://username:password@gitea.com",
`<p><a href="https://username:password@gitea.com" rel="nofollow">https://username:password@gitea.com</a></p>`)
test(
"ftp://gitea.com/file.txt",
`<p><a href="ftp://gitea.com/file.txt" rel="nofollow">ftp://gitea.com/file.txt</a></p>`)
test(
"magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&dn=download",
`<p><a href="magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download" rel="nofollow">magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download</a></p>`)
test(
`[link](https://example.com)`,
`<p><a href="https://example.com" rel="nofollow">link</a></p>`)
test(
`[link](mailto:test@example.com)`,
`<p><a href="mailto:test@example.com" rel="nofollow">link</a></p>`)
test(
`[link](javascript:xss)`,
`<p>link</p>`)
// Test that should *not* be turned into URL
test(
"www.example.com",
`<p>www.example.com</p>`)
test(
"example.com",
`<p>example.com</p>`)
test(
"test.example.com",
`<p>test.example.com</p>`)
test(
"http://",
`<p>http://</p>`)
test(
"https://",
`<p>https://</p>`)
test(
"://",
`<p>://</p>`)
test(
"www",
`<p>www</p>`)
test(
"ftps://gitea.com",
`<p>ftps://gitea.com</p>`)
}
func TestRender_email(t *testing.T) {
setting.AppURL = markup.TestAppURL
test := func(input, expected string) {
res, err := markup.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
RelativePath: "a.md",
Links: markup.Links{
Base: markup.TestRepoURL,
},
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(res))
}
// Text that should be turned into email link
test(
"info@gitea.com",
`<p><a href="mailto:info@gitea.com" rel="nofollow">info@gitea.com</a></p>`)
test(
"(info@gitea.com)",
`<p>(<a href="mailto:info@gitea.com" rel="nofollow">info@gitea.com</a>)</p>`)
test(
"[info@gitea.com]",
`<p>[<a href="mailto:info@gitea.com" rel="nofollow">info@gitea.com</a>]</p>`)
test(
"info@gitea.com.",
`<p><a href="mailto:info@gitea.com" rel="nofollow">info@gitea.com</a>.</p>`)
test(
"firstname+lastname@gitea.com",
`<p><a href="mailto:firstname+lastname@gitea.com" rel="nofollow">firstname+lastname@gitea.com</a></p>`)
test(
"send email to info@gitea.co.uk.",
`<p>send email to <a href="mailto:info@gitea.co.uk" rel="nofollow">info@gitea.co.uk</a>.</p>`)
test(
`j.doe@example.com,
j.doe@example.com.
j.doe@example.com;
j.doe@example.com?
j.doe@example.com!`,
`<p><a href="mailto:j.doe@example.com" rel="nofollow">j.doe@example.com</a>,<br/>
<a href="mailto:j.doe@example.com" rel="nofollow">j.doe@example.com</a>.<br/>
<a href="mailto:j.doe@example.com" rel="nofollow">j.doe@example.com</a>;<br/>
<a href="mailto:j.doe@example.com" rel="nofollow">j.doe@example.com</a>?<br/>
<a href="mailto:j.doe@example.com" rel="nofollow">j.doe@example.com</a>!</p>`)
// Test that should *not* be turned into email links
test(
"\"info@gitea.com\"",
`<p>&#34;info@gitea.com&#34;</p>`)
test(
"/home/gitea/mailstore/info@gitea/com",
`<p>/home/gitea/mailstore/info@gitea/com</p>`)
test(
"git@try.gitea.io:go-gitea/gitea.git",
`<p>git@try.gitea.io:go-gitea/gitea.git</p>`)
test(
"gitea@3",
`<p>gitea@3</p>`)
test(
"gitea@gmail.c",
`<p>gitea@gmail.c</p>`)
test(
"email@domain@domain.com",
`<p>email@domain@domain.com</p>`)
test(
"email@domain..com",
`<p>email@domain..com</p>`)
}
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
func TestRender_emoji(t *testing.T) {
setting.AppURL = markup.TestAppURL
setting.StaticURLPrefix = markup.TestAppURL
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
test := func(input, expected string) {
expected = strings.ReplaceAll(expected, "&", "&amp;")
buffer, err := markup.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
RelativePath: "a.md",
Links: markup.Links{
Base: markup.TestRepoURL,
},
}, input)
assert.NoError(t, err)
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
}
// Make sure we can successfully match every emoji in our dataset with regex
for i := range emoji.GemojiData {
test(
emoji.GemojiData[i].Emoji,
`<p><span class="emoji" aria-label="`+emoji.GemojiData[i].Description+`">`+emoji.GemojiData[i].Emoji+`</span></p>`)
}
for i := range emoji.GemojiData {
test(
":"+emoji.GemojiData[i].Aliases[0]+":",
`<p><span class="emoji" aria-label="`+emoji.GemojiData[i].Description+`">`+emoji.GemojiData[i].Emoji+`</span></p>`)
}
// Text that should be turned into or recognized as emoji
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
test(
":gitea:",
`<p><span class="emoji" aria-label="gitea"><img alt=":gitea:" src="`+setting.StaticURLPrefix+`/assets/img/emoji/gitea.png"/></span></p>`)
2021-06-29 08:28:38 -06:00
test(
":custom-emoji:",
`<p>:custom-emoji:</p>`)
setting.UI.CustomEmojisMap["custom-emoji"] = ":custom-emoji:"
test(
":custom-emoji:",
`<p><span class="emoji" aria-label="custom-emoji"><img alt=":custom-emoji:" src="`+setting.StaticURLPrefix+`/assets/img/emoji/custom-emoji.png"/></span></p>`)
test(
"这是字符:1::+1: some🐊 \U0001f44d:custom-emoji: :gitea:",
`<p>这是字符:1:<span class="emoji" aria-label="thumbs up">👍</span> some<span class="emoji" aria-label="crocodile">🐊</span> `+
`<span class="emoji" aria-label="thumbs up">👍</span><span class="emoji" aria-label="custom-emoji"><img alt=":custom-emoji:" src="`+setting.StaticURLPrefix+`/assets/img/emoji/custom-emoji.png"/></span> `+
`<span class="emoji" aria-label="gitea"><img alt=":gitea:" src="`+setting.StaticURLPrefix+`/assets/img/emoji/gitea.png"/></span></p>`)
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
test(
"Some text with 😄 in the middle",
`<p>Some text with <span class="emoji" aria-label="grinning face with smiling eyes">😄</span> in the middle</p>`)
test(
"Some text with :smile: in the middle",
`<p>Some text with <span class="emoji" aria-label="grinning face with smiling eyes">😄</span> in the middle</p>`)
test(
"Some text with 😄😄 2 emoji next to each other",
`<p>Some text with <span class="emoji" aria-label="grinning face with smiling eyes">😄</span><span class="emoji" aria-label="grinning face with smiling eyes">😄</span> 2 emoji next to each other</p>`)
test(
"😎🤪🔐🤑❓",
`<p><span class="emoji" aria-label="smiling face with sunglasses">😎</span><span class="emoji" aria-label="zany face">🤪</span><span class="emoji" aria-label="locked with key">🔐</span><span class="emoji" aria-label="money-mouth face">🤑</span><span class="emoji" aria-label="red question mark">❓</span></p>`)
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 12:05:39 -06:00
// should match nothing
test(
"2001:0db8:85a3:0000:0000:8a2e:0370:7334",
`<p>2001:0db8:85a3:0000:0000:8a2e:0370:7334</p>`)
test(
":not exist:",
`<p>:not exist:</p>`)
}
func TestRender_ShortLinks(t *testing.T) {
setting.AppURL = markup.TestAppURL
tree := util.URLJoin(markup.TestRepoURL, "src", "master")
test := func(input, expected, expectedWiki string) {
buffer, err := markdown.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
Base: markup.TestRepoURL,
BranchPath: "master",
},
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(string(buffer)))
buffer, err = markdown.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
Base: markup.TestRepoURL,
},
Metas: localMetas,
IsWiki: true,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expectedWiki), strings.TrimSpace(string(buffer)))
}
mediatree := util.URLJoin(markup.TestRepoURL, "media", "master")
url := util.URLJoin(tree, "Link")
otherURL := util.URLJoin(tree, "Other-Link")
2018-06-15 06:42:49 -06:00
encodedURL := util.URLJoin(tree, "Link%3F")
imgurl := util.URLJoin(mediatree, "Link.jpg")
otherImgurl := util.URLJoin(mediatree, "Link+Other.jpg")
encodedImgurl := util.URLJoin(mediatree, "Link+%23.jpg")
notencodedImgurl := util.URLJoin(mediatree, "some", "path", "Link+#.jpg")
urlWiki := util.URLJoin(markup.TestRepoURL, "wiki", "Link")
otherURLWiki := util.URLJoin(markup.TestRepoURL, "wiki", "Other-Link")
encodedURLWiki := util.URLJoin(markup.TestRepoURL, "wiki", "Link%3F")
imgurlWiki := util.URLJoin(markup.TestRepoURL, "wiki", "raw", "Link.jpg")
otherImgurlWiki := util.URLJoin(markup.TestRepoURL, "wiki", "raw", "Link+Other.jpg")
encodedImgurlWiki := util.URLJoin(markup.TestRepoURL, "wiki", "raw", "Link+%23.jpg")
notencodedImgurlWiki := util.URLJoin(markup.TestRepoURL, "wiki", "raw", "some", "path", "Link+#.jpg")
renderableFileURL := util.URLJoin(tree, "markdown_file.md")
renderableFileURLWiki := util.URLJoin(markup.TestRepoURL, "wiki", "markdown_file.md")
unrenderableFileURL := util.URLJoin(tree, "file.zip")
unrenderableFileURLWiki := util.URLJoin(markup.TestRepoURL, "wiki", "raw", "file.zip")
favicon := "http://google.com/favicon.ico"
test(
"[[Link]]",
`<p><a href="`+url+`" rel="nofollow">Link</a></p>`,
`<p><a href="`+urlWiki+`" rel="nofollow">Link</a></p>`)
test(
"[[Link.-]]",
`<p><a href="http://localhost:3000/test-owner/test-repo/src/master/Link.-" rel="nofollow">Link.-</a></p>`,
`<p><a href="http://localhost:3000/test-owner/test-repo/wiki/Link.-" rel="nofollow">Link.-</a></p>`)
test(
"[[Link.jpg]]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="Link.jpg" alt="Link.jpg"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="Link.jpg" alt="Link.jpg"/></a></p>`)
test(
"[["+favicon+"]]",
`<p><a href="`+favicon+`" rel="nofollow"><img src="`+favicon+`" title="favicon.ico" alt="`+favicon+`"/></a></p>`,
`<p><a href="`+favicon+`" rel="nofollow"><img src="`+favicon+`" title="favicon.ico" alt="`+favicon+`"/></a></p>`)
test(
"[[Name|Link]]",
`<p><a href="`+url+`" rel="nofollow">Name</a></p>`,
`<p><a href="`+urlWiki+`" rel="nofollow">Name</a></p>`)
test(
"[[Name|Link.jpg]]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="Name" alt="Name"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="Name" alt="Name"/></a></p>`)
test(
"[[Name|Link.jpg|alt=AltName]]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="AltName" alt="AltName"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="AltName" alt="AltName"/></a></p>`)
test(
"[[Name|Link.jpg|title=Title]]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="Title" alt="Title"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="Title" alt="Title"/></a></p>`)
test(
"[[Name|Link.jpg|alt=AltName|title=Title]]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="Title" alt="AltName"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="Title" alt="AltName"/></a></p>`)
test(
"[[Name|Link.jpg|alt=\"AltName\"|title='Title']]",
`<p><a href="`+imgurl+`" rel="nofollow"><img src="`+imgurl+`" title="Title" alt="AltName"/></a></p>`,
`<p><a href="`+imgurlWiki+`" rel="nofollow"><img src="`+imgurlWiki+`" title="Title" alt="AltName"/></a></p>`)
test(
"[[Name|Link Other.jpg|alt=\"AltName\"|title='Title']]",
`<p><a href="`+otherImgurl+`" rel="nofollow"><img src="`+otherImgurl+`" title="Title" alt="AltName"/></a></p>`,
`<p><a href="`+otherImgurlWiki+`" rel="nofollow"><img src="`+otherImgurlWiki+`" title="Title" alt="AltName"/></a></p>`)
test(
"[[Link]] [[Other Link]]",
`<p><a href="`+url+`" rel="nofollow">Link</a> <a href="`+otherURL+`" rel="nofollow">Other Link</a></p>`,
`<p><a href="`+urlWiki+`" rel="nofollow">Link</a> <a href="`+otherURLWiki+`" rel="nofollow">Other Link</a></p>`)
2018-06-15 06:42:49 -06:00
test(
"[[Link?]]",
`<p><a href="`+encodedURL+`" rel="nofollow">Link?</a></p>`,
`<p><a href="`+encodedURLWiki+`" rel="nofollow">Link?</a></p>`)
test(
"[[Link]] [[Other Link]] [[Link?]]",
`<p><a href="`+url+`" rel="nofollow">Link</a> <a href="`+otherURL+`" rel="nofollow">Other Link</a> <a href="`+encodedURL+`" rel="nofollow">Link?</a></p>`,
`<p><a href="`+urlWiki+`" rel="nofollow">Link</a> <a href="`+otherURLWiki+`" rel="nofollow">Other Link</a> <a href="`+encodedURLWiki+`" rel="nofollow">Link?</a></p>`)
test(
"[[markdown_file.md]]",
`<p><a href="`+renderableFileURL+`" rel="nofollow">markdown_file.md</a></p>`,
`<p><a href="`+renderableFileURLWiki+`" rel="nofollow">markdown_file.md</a></p>`)
test(
"[[file.zip]]",
`<p><a href="`+unrenderableFileURL+`" rel="nofollow">file.zip</a></p>`,
`<p><a href="`+unrenderableFileURLWiki+`" rel="nofollow">file.zip</a></p>`)
2018-06-15 06:42:49 -06:00
test(
"[[Link #.jpg]]",
`<p><a href="`+encodedImgurl+`" rel="nofollow"><img src="`+encodedImgurl+`" title="Link #.jpg" alt="Link #.jpg"/></a></p>`,
`<p><a href="`+encodedImgurlWiki+`" rel="nofollow"><img src="`+encodedImgurlWiki+`" title="Link #.jpg" alt="Link #.jpg"/></a></p>`)
2018-06-15 06:42:49 -06:00
test(
"[[Name|Link #.jpg|alt=\"AltName\"|title='Title']]",
`<p><a href="`+encodedImgurl+`" rel="nofollow"><img src="`+encodedImgurl+`" title="Title" alt="AltName"/></a></p>`,
`<p><a href="`+encodedImgurlWiki+`" rel="nofollow"><img src="`+encodedImgurlWiki+`" title="Title" alt="AltName"/></a></p>`)
test(
"[[some/path/Link #.jpg]]",
`<p><a href="`+notencodedImgurl+`" rel="nofollow"><img src="`+notencodedImgurl+`" title="Link #.jpg" alt="some/path/Link #.jpg"/></a></p>`,
`<p><a href="`+notencodedImgurlWiki+`" rel="nofollow"><img src="`+notencodedImgurlWiki+`" title="Link #.jpg" alt="some/path/Link #.jpg"/></a></p>`)
test(
"<p><a href=\"https://example.org\">[[foobar]]</a></p>",
`<p><a href="https://example.org" rel="nofollow">[[foobar]]</a></p>`,
`<p><a href="https://example.org" rel="nofollow">[[foobar]]</a></p>`)
}
func TestRender_RelativeImages(t *testing.T) {
render := func(input string, isWiki bool, links markup.Links) string {
buffer, err := markdown.RenderString(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: links,
Metas: localMetas,
IsWiki: isWiki,
}, input)
assert.NoError(t, err)
return strings.TrimSpace(string(buffer))
}
out := render(`<img src="LINK">`, false, markup.Links{Base: "/test-owner/test-repo"})
assert.Equal(t, `<a href="/test-owner/test-repo/LINK" target="_blank" rel="nofollow noopener"><img src="/test-owner/test-repo/LINK"/></a>`, out)
out = render(`<img src="LINK">`, true, markup.Links{Base: "/test-owner/test-repo"})
assert.Equal(t, `<a href="/test-owner/test-repo/wiki/raw/LINK" target="_blank" rel="nofollow noopener"><img src="/test-owner/test-repo/wiki/raw/LINK"/></a>`, out)
out = render(`<img src="LINK">`, false, markup.Links{Base: "/test-owner/test-repo", BranchPath: "test-branch"})
assert.Equal(t, `<a href="/test-owner/test-repo/media/test-branch/LINK" target="_blank" rel="nofollow noopener"><img src="/test-owner/test-repo/media/test-branch/LINK"/></a>`, out)
out = render(`<img src="LINK">`, true, markup.Links{Base: "/test-owner/test-repo", BranchPath: "test-branch"})
assert.Equal(t, `<a href="/test-owner/test-repo/wiki/raw/LINK" target="_blank" rel="nofollow noopener"><img src="/test-owner/test-repo/wiki/raw/LINK"/></a>`, out)
out = render(`<img src="/LINK">`, true, markup.Links{Base: "/test-owner/test-repo", BranchPath: "test-branch"})
assert.Equal(t, `<img src="/LINK"/>`, out)
}
func Test_ParseClusterFuzz(t *testing.T) {
setting.AppURL = markup.TestAppURL
localMetas := map[string]string{
"user": "go-gitea",
"repo": "gitea",
}
data := "<A><maTH><tr><MN><bodY ÿ><temPlate></template><tH><tr></A><tH><d<bodY "
var res strings.Builder
err := markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
Base: "https://example.com",
},
Metas: localMetas,
}, strings.NewReader(data), &res)
assert.NoError(t, err)
assert.NotContains(t, res.String(), "<html")
data = "<!DOCTYPE html>\n<A><maTH><tr><MN><bodY ÿ><temPlate></template><tH><tr></A><tH><d<bodY "
res.Reset()
err = markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
Base: "https://example.com",
},
Metas: localMetas,
}, strings.NewReader(data), &res)
assert.NoError(t, err)
assert.NotContains(t, res.String(), "<html")
}
func TestPostProcess_RenderDocument(t *testing.T) {
setting.AppURL = markup.TestAppURL
setting.StaticURLPrefix = markup.TestAppURL // can't run standalone
localMetas := map[string]string{
"user": "go-gitea",
"repo": "gitea",
"mode": "document",
}
test := func(input, expected string) {
var res strings.Builder
err := markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
AbsolutePrefix: true,
Base: "https://example.com",
},
Metas: localMetas,
}, strings.NewReader(input), &res)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(res.String()))
}
// Issue index shouldn't be post processing in a document.
test(
"#1",
"#1")
// But cross-referenced issue index should work.
test(
"go-gitea/gitea#12345",
`<a href="`+util.URLJoin(markup.TestAppURL, "go-gitea", "gitea", "issues", "12345")+`" class="ref-issue">go-gitea/gitea#12345</a>`)
// Test that other post processing still works.
test(
":gitea:",
`<span class="emoji" aria-label="gitea"><img alt=":gitea:" src="`+setting.StaticURLPrefix+`/assets/img/emoji/gitea.png"/></span>`)
test(
"Some text with 😄 in the middle",
`Some text with <span class="emoji" aria-label="grinning face with smiling eyes">😄</span> in the middle`)
test("http://localhost:3000/person/repo/issues/4#issuecomment-1234",
`<a href="http://localhost:3000/person/repo/issues/4#issuecomment-1234" class="ref-issue">person/repo#4 (comment)</a>`)
}
func TestIssue16020(t *testing.T) {
setting.AppURL = markup.TestAppURL
localMetas := map[string]string{
"user": "go-gitea",
"repo": "gitea",
}
data := `<img src="data:image/png;base64,i//V"/>`
var res strings.Builder
err := markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Metas: localMetas,
}, strings.NewReader(data), &res)
assert.NoError(t, err)
assert.Equal(t, data, res.String())
}
func BenchmarkEmojiPostprocess(b *testing.B) {
data := "🥰 "
for len(data) < 1<<16 {
data += data
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
var res strings.Builder
err := markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Metas: localMetas,
}, strings.NewReader(data), &res)
assert.NoError(b, err)
}
}
func TestFuzz(t *testing.T) {
s := "t/l/issues/8#/../../a"
renderContext := markup.RenderContext{
Ctx: git.DefaultContext,
Links: markup.Links{
Base: "https://example.com/go-gitea/gitea",
},
Metas: map[string]string{
"user": "go-gitea",
"repo": "gitea",
},
}
err := markup.PostProcess(&renderContext, strings.NewReader(s), io.Discard)
assert.NoError(t, err)
}
func TestIssue18471(t *testing.T) {
data := `http://domain/org/repo/compare/783b039...da951ce`
var res strings.Builder
err := markup.PostProcess(&markup.RenderContext{
Ctx: git.DefaultContext,
Metas: localMetas,
}, strings.NewReader(data), &res)
assert.NoError(t, err)
assert.Equal(t, `<a href="http://domain/org/repo/compare/783b039...da951ce" class="compare"><code class="nohighlight">783b039...da951ce</code></a>`, res.String())
}
func TestIsFullURL(t *testing.T) {
assert.True(t, markup.IsFullURLString("https://example.com"))
assert.True(t, markup.IsFullURLString("mailto:test@example.com"))
assert.True(t, markup.IsFullURLString("data:image/11111"))
assert.False(t, markup.IsFullURLString("/foo:bar"))
}