Within Codeberg we are looking into distributing the database queries, we tried forgejo/forgejo!7212 on several occasions but never got it to work.
After a long debugging session in a staging environment I was able to find two bugs that made it impossible for this feature to work: forgejo/docs!1587 which resulted in replica engines never being configured and used if you followed the documentation. The other bug is what this patch intends to fix. In order to do some database operation, you need the database engine - it will first look if one is set for the context (only useful for transactions) and otherwise create a new session of the engine from the master engine `x`. The problem is that `x` is explicitly set to be the master engine and not the engine group (that includes the replica engines) - Unless the code uses `DefaultContext`, which is almost nowhere used after some great refactoring in Gitea to use the passed context, it did not use the replica engines.
Get engine from the `DefaultContext` (which is set to the enginegroup) and create a new session from that.
https://codeberg.org/forgejo/forgejo/src/commit/20f8572b923c83258a53f1629782953000dd0e75/models/db/engine.go#L220-L231
And `SetDefaultEngine` is called from https://codeberg.org/forgejo/forgejo/src/commit/20f8572b923c83258a53f1629782953000dd0e75/models/db/engine.go#L212
Where `eng` is the engine group.
## Test
1. Configure database replicas.
2. Start Forgejo.
3. Verify Forgejo loads.
4. Stop the database replicas.
5. Verify Forgejo shows 500 errors.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10140
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Although #9922 was deployed to Codeberg, it was reported on Matrix that a user observed a `-1` pull request count.
@Gusted checked and verified that the stats stored in redis appeared incorrect, and that no errors occurred on Codeberg that included the repo ID (eg. deadlocks, SQL queries).
```
127.0.0.1:6379> GET Repo:CountPulls:924266
"1"
127.0.0.1:6379> GET Repo:CountPullsClosed:924266
"2"
```
One possible cause is that when `UpdateRepoIssueNumbers` is invoked and invalidates the cache key for the repository, it is currently in a transaction; the next request for that cached count could be computed before the transaction is committed and the update is visible. It's been verified that `UpdateRepoIssueNumbers` is called within a transaction in most interactions (I put a panic in it if `db.InTransaction(ctx)`, and most related tests failed).
This PR fixes that hole by performing the cache invalidation in an `AfterTx()` hook which is invoked after the transaction is committed to the database.
(Another possible cause is documented in #10127)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10130
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
`modules/queue` tests are intended to be running against a redis server in the test-remote-cacher workflow action, but they are not running because the tests always try to start a redis server by running a `redis-server` subprocess. If that subprocess fails to start, the tests are skipped.
This change forces the tests to execute whenever `TEST_REDIS_SERVER` is present, and removes the unnecessary Forgejo-managed redis server during testing.
This change is tested manually; if `TEST_REDIS_SERVER` is set to an invalid value, the tests fail, when previously it could be set to any value and the tests would always be skipped.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10139
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Resolves#10102
Adjust the SSH key verification instructions for Windows CMD to make it work.
## Test
1. Use Windows.
2. Go to the user settings.
3. Go to "SSH / GPG keys"
4. Add a SSH key.
5. Click on the "Verify" button for the SSH key.
6. Use the Windows CMD command to verify.
7. Verify that this generates a signature that is accepted by Forgejo.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10106
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Melissa Geels <melissa@nimble.tools>
Co-committed-by: Melissa Geels <melissa@nimble.tools>
Replaced dropdowns in the navbar with JS-less ones from https://codeberg.org/forgejo/forgejo/pulls/7906.
Also made some changes to the dropdown component:
* fixed variable name
* painted backgrounds (hover, focus) are now consistently applied to the actual interactive items (`<a>`, `<button>`), not to `<li>`. This is consistent with how backgrounds are conditionally applied to pre-selected (`.active`) items and is better, as it allows to place additional things to `<li>`...
* ...`<hr>` can now be placed in some `<li>` instead of requiring splitting into multiple `<ul>`. This is simpler in code and I am guessing this should be better for a11y as screen readers can cast one continuous list instead of multiple ones. But have no hard proof that this is actually better. My main motivation was to avoid ugly mistake-prone tmpl logic where unconditional `<ul>` was getting closed and reopened inside of a condition.
I should note that on mobile all items, including these dropdowns, are hidden in another dropdown, and it stays JS-dependand for now. So this PR only makes this part of the UI JS-less for desktop.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10025
Reviewed-by: Robert Wolff <mahlzahn@posteo.de>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: 0ko <0ko@noreply.codeberg.org>
Co-committed-by: 0ko <0ko@noreply.codeberg.org>
Currently `DeriveKey` is called every time that a secret must be encoded/decoded. Since this function is deterministic, its result can be cached to allow a 250x speedup (the original took less than half a microsecond, so this more of a micro-optimization...).
```
go test -bench=.
goos: linux
goarch: amd64
pkg: forgejo.org/modules/keying
cpu: Intel(R) Core(TM) Ultra 5 125H
BenchmarkExpandPRK-18 2071627 564.2 ns/op
BenchmarkExpandPRKOnce-18 541438192 2.206 ns/op
PASS
ok forgejo.org/modules/keying 2.369s
```
## Other changes
- Since the keys can be constructed once, it simplifies a bit the callsites (`keying.TOTP.Encrypt(...)` instead of `keying.DeriveKey(keying.ContextTOTP).Encrypt(...)`)
- All `Encrypt`/`Decrypt` calls will panic forever if called before `Init` has been called (current it panics as long as `Init` has not been called)
- Calling `Init` twice with different keys will trigger a panic (currently racy)
- Calling `Decrypt` with a short ciphertext does not panic anymore (like when calling with long-enough garbage)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10114
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Move the function to the repository struct. There is no need to have it as a separate function, move it to the Repository struct. Add extra unit tests.
---
Remove a field from a struct. It has nothing to do with git, it is not the right place to have that field in the git `Tag` struct. Get this value when it's converted to the API struct.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10109
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
See #8222 for context (loosely related to #4595).
## Implemented changes
The conversion logic is kept in the frontend and the related npm libraries are lazy-loaded (unchanged).
### Show some tabs on the preview of the `CITATION.*` file to switch between the formats:
### Convert the "Cite repository" to a simple link to the citation file
So that this change can be considered non-breaking
## Current state (before this PR)
The last non-test call of `git.Blob.GetBlobContent` is made to retrieve the content of an eventual CITATION file.
This is available in the `...` menu near the clone URL:
And is displayed as a popup:
Co-authored-by: 0ko <0ko@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9103
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Webkit and Mobile safari are comically unreliable, will fail for unexplainable reasons and are very hard to run locally in comparison with the other supported platforms. I do not remember the last time where these two platforms were able to catch a regression where the other platforms did not.
I would like to stress, for the historical record, that many hours has been devoted into adjusting the tests and following best practices to make these two platforms more stable but despite those, IMO wasted, efforts these two platforms are causing many hours of wasted CPU time simply because they are flaky and make (new) contributors nervous if their change contains a regression or not.
To my knowledge, the tests are not broken for these two platforms. If you go to the issue tracker you will not find issues by users that use these two platforms and report that Forgejo is broken. It does not reflect reality.
This is the sunk cost fallacy, bite the bullet and agree that these platforms will not contribute positively to Forgejo's excellent test suite.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10103
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Forgejo Actions allows variables in `jobs.<job_id>.runs-on`. However, the action list [skips checking whether a suitable runner](https://codeberg.org/forgejo/forgejo/src/commit/c3412d0579acfc702224f1785fc44e5695daee90/routers/web/repo/actions/actions.go#L114-L148) is available if an expression contains variables. That hampers a user's ability to figure out whether an expression was evaluated correctly and why a job might not be picked up by an available runner.
This PR adds the ability to surface more complex and additional diagnostic information on the action view screen. Previously, only a job's status (waiting, running, ...) was displayed. Now, extended messages like "Waiting for a runner with the following labels: docker, trixie" are displayed with the possibility to show multiple messages simultaneously.
How it looked before:
How it looks after updating Forgejo without reloading the window:
How it looks afterwards with a single label:
How it looks afterwards with multiple labels:
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [x] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/9966): <!--number 9966 --><!--line 0 --><!--description ZGlzcGxheSBkZXRhaWxlZCBhY3Rpb24gcnVuIGRpYWdub3N0aWNz-->display detailed action run diagnostics<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9966
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Previously, the variable value was empty when asking for a list of all repository variables.
I have extended the test coverage of all related endpoints and made the existing tests independent of each other.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Other changes without a feature or bug label
- [PR](https://codeberg.org/forgejo/forgejo/pulls/10036): <!--number 10036 --><!--line 0 --><!--description aW5jbHVkZSB2YXJpYWJsZSB2YWx1ZXMgaW4gL3JlcG9zLy4uLi9hY3Rpb25zL3ZhcmlhYmxlcyBBUEkgcmVzcG9uc2U=-->include variable values in /repos/.../actions/variables API response<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10036
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
Co-committed-by: Andreas Ahlenstorf <andreas@ahlenstorf.ch>
In forgejo/forgejo!2834 and forgejo/forgejo!5307 it was made so it's no longer possible to modify and delete internal reference, not having this restriction lead to broken pull requests when people used something like `git push --mirror`. However it now still leads to problem with that command as the git client tries to delete such references. We can solve this by using git's `receive.hideRefs` to make this ref read-only and avoid advertising it when someone does `git push --mirror`.
Resolvesforgejo/forgejo#9942
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10015
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Signed-off-by: voltagex <git@voltagex.org>
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10035
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: voltagex <git@voltagex.org>
Co-committed-by: voltagex <git@voltagex.org>
