Allow authentication to git HTTP & git LFS via an authorized integration.
This is the first step in getting rid of OAuth, basic auth, etc.'s usage of [`isGitRawOrAttachPath(req)`](https://codeberg.org/forgejo/forgejo/src/commit/26f18a94ee80a5fd8972833b95b1df995ee92fc4/services/auth/method/basic.go#L38-L40). I don't want to follow that pattern of HTTP route matching in the authentication method, so I've broken the HTTP routes related to git functionality out to using a separate authentication middleware in the top-level `web.Routes` handler. As this approach is expanded to the other endpoints in order to add support to them for authorized integrations, eventually it will be possible to remove this URL matching completely and just rely on middleware installation.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12715
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Adds a "Delete" option to the authorized integration list.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [x] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12632
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Extends the UI introduced in #12558 to have edit capabilities. (not in scope: "Add" for a new Authorized Integration will be the next update to this UI; `create-authorized-integration` CLI is still the only way to create a new record)
This PR includes a few refactoring steps. The goal of these steps is to have `services/auth` be a single entrypoint for validating, inserting, or updating an authorized integration. Some logic is moved out of `services/authz` because it is not authorization related, and some is moved out of `services/auth/method` to allow it to be reused during validation without creating a cyclical module dependency.
This PR also adds comprehensive validation to the more complex fields in the authorized integration, such as the issuer and claim rules. This validation applies to the `forgejo admin user create-authorized-integration` CLI as well.
The visible UI is the same as #12558, but with a "Save" button, and the ability to display errors:
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [x] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
- Documentation is on my TODO list and will be completed before release.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12601
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
The session cookie has no Max-Age, so it is lost when the browser closes. The password flow compensates via a "Remember me" checkbox issuing an LTA cookie; OAuth2/OIDC sign-in had no such UI.
Issuing a regular LTA cookie after an OAuth callback would skip the IdP for LOGIN_REMEMBER_DAYS. Instead, this introduces a separate LongTermAuthorizationSSO purpose: the cookie is opt-in via the existing "Remember me" checkbox, and when presented without a session, autoSignIn redirects through the IdP with OIDC prompt=none for silent re-auth. On login_required / interaction_required / consent_required / account_selection_required
we transparently fall back to interactive sign-in.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12321
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
User interfaces for authorized integrations will benefit from having a name field, to allow a list of authorized integrations to have an identifiable user-entered label.
I've also added a "description" column which is a `LONGTEXT` field. My thought for this field is that if I were creating authorized integrations, I'd like to be able to write down where they're used, what they're used for, and how the remote system is configured. For example, if it was an authorized integration to allow AWS -> Forgejo integration, the AWS side can be complicated -- IAM roles which are assumed, resources like EC2 instances or Lambdas that can access the roles -- and this would provide a natural place to make some notes to help me remember how the remote is configured. I expect to represent this as a `<textarea>` in the Authorized Integration, optional, possibly markdown-formatted to allow links & bullet-points.
Manually tested migration with PG backend, and manually tested creation of authorized integrations with the CLI updates.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12413
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Allows the creation of an authorized integration as a Forgejo administrator, either for development testing or to support server-automation. Clipping out the CLI config options, looks like:
```
NAME:
forgejo admin user create-authorized-integration - Create an authorized integration for a specific user
USAGE:
forgejo admin user create-authorized-integration [options]
OPTIONS:
--username string, -u string Username
--issuer string JWT issuer ('iss' claim), example: https://forgejo.example.org/api/actions
--claim-eq string=string [ --claim-eq string=string ] Zero-or-more claim equality checks, formatted as claim=value, example: "actor=someuser"
--claim-glob string=string [ --claim-glob string=string ] Zero-or-more claim glob checks, formatted as claim=value, example: "sub=repo:forgejo/*:pull_request"
--scope string [ --scope string ] One-or-more scopes to apply to access token, examples: "all", "read:issue", "write:repository" (default: "all")
--repo string [ --repo string ] Zero-or-more specific repositories that can be accessed, or "all" to allow access to all repositories, example: "owner1/repo1" (default: "all")
```
As an example, this will create an authorized integration that will permit Codeberg's Forgejo Actions to generate trusted JWTs that can access the local user `mfenniak`:
```bash
$ ./forgejo admin user create-authorized-integration \
--username mfenniak \
--issuer https://codeberg.org/api/actions \
--claim-eq sub=repo:mfenniak/forgejo-runner-testrepo:pull_request \
--scope read:user
{
"message": "Authorized integration was successfully created.",
"issuer": "https://codeberg.org/api/actions",
"audience": "u:1:c97d83bc-fa4e-4db3-b898-414cd5b6ce33",
"claim_rules": [
{
"description": "\"sub\" = \"repo:mfenniak/forgejo-runner-testrepo:pull_request\"",
"claim": "sub",
"compare": "eq",
"value": "repo:mfenniak/forgejo-runner-testrepo:pull_request"
}
]
}
```
The output is a JSON document to aid in use in automation. The `audience` field is the audience generated by Forgejo that must be used by the remote to generate the JWT. Continuing this example to the client-side, a matching Forgejo Action like this in the `mfenniak/forgejo-runner-testrepo` repo, for a `pull_request` event, then it will be able to access the Forgejo server that the authorized integration was created on like this:
```yaml
on:
pull_request:
enable-openid-connect: true
jobs:
job1:
runs-on: docker
steps:
- name: Fetch JWT
id: jwt
run: |
set -eux -o pipefail
set +x
jwt=$(curl --fail \
-H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=u:1:c97d83bc-fa4e-4db3-b898-414cd5b6ce33" \
| jq -r ".value")
echo "::add-mask::$jwt"
set -x
echo "jwt=$jwt" >> $FORGEJO_OUTPUT
- name: API call to Forgejo
run: |
curl \
-v --fail \
-H "Authorization: bearer ${{ steps.jwt.outputs.jwt }}" \
"https://example.org/api/v1/user" | jq
```
CLI command is tested manually. Supporting functions have associated unit tests.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
- CLI update should be automatic in docs -- more detailed Authorized Integration documentation is on my project plan.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12299
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Built on #12266; one commit added.
Adds the ability to reduce the authorization scope of an authorized integration to public-only resources and repo-specific resources. Backend only -- no frontend created yet.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12267
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Authorized Integrations is a new feature to allow users to define external systems which can generate JSON Web Tokens (JWTs) that Forgejo will trust in order to perform API access on behalf of that user. This is an authentication mechanism that requires zero preconfiguration of shared secrets, and instead establishes trust through short-lived secrets (JWTs) that are signed by the issuer, signatures are validated by comparison with published public keys, and a public-keys retrieved through well-known HTTP endpoints secured with TLS verification.
The primary goal of Authorized Integrations is to support a mechanism for Forgejo Actions to receive elevated, but controlled, additional access to Forgejo. More details as to what the end result will look like are available in the [design proposal](https://codeberg.org/forgejo/forgejo/issues/3571#issuecomment-13268004) on #3571.
This PR adds the core database storage and authentication verification for Authorized Integrations, with these capabilities:
- An Authorized Integration is resolved by a unique key of an "issuer" and an "audience". The value of "issuer" is defined by the remote integration, and the value of "audience" will incorporate a unique identifier generated by Forgejo.
- Example issuer: `https://token.actions.githubusercontent.com/` is the issuer for GitHub JWTs
- Example audience: `https://forgejo.example.org/-/mfenniak/authorized-integration/6cc55ba0` is the expected format for a random audience field that Forgejo will generate.
- JWTs can contain any number of claims, which are represented as a JSON object; Forgejo can validate these with a flexible policy.
- eg. a claim may be `{"sub": "repo:coolguy/forgejo-runner-testrepo:pull_request"}` indicating that an OIDC token was received from an Actions execution in a specific repo on a specific event.
- Authorized Integrations support a `ClaimRules` system which allows claim equal, glob, and nested object inspection.
- `{"claim":"sub","comparison":"eq","value":"repo:mfenniak/forgejo-runner-testrepo:pull_request"}` -- would validate that `sub` exactly equals the specific value
- `{"claim":"sub","comparison":"glob","value":"repo:mfenniak/forgejo-runner-testrepo:*"}` -- would validate that `sub` matches the given string prefix but allow any event
- When a JWT is received on an incoming API call, Forgejo retrieves the Authorized Integration from the DB (if present), validates the token signature against a remote JWKS, validates the claims, and grants API access as the user with a permission scope defined on the Authorized Integration.
In addition to the unit testing provided here, this PR has been manually integration tested against three JWT issuing systems: Forgejo Actions, GitHub Actions, and AWS STS GetWebIdentityToken.
Careful consideration has been made of these security concerns:
- SSRF attacks against Forgejo are prevented by:
- having a blocklist on remote HTTP validation requests which prevent access to internal network resources,
- ensuring that authorized integrations are created by users with matching issuers, before attempting to validate tokens
- Resource utilization attacks against Forgejo are reduced by limiting the possible size of external metadata requests; when fetching `/.well-known/openid-configuration` and `jkws_uri`'s from remote, untrusted servers, a maximum response size of 16 kB is enforced
- Only well-known secure assymmetric JWT signing algorithms are supported -- in particular, the sketchy `none` JWT algorithm isn't supported.
- JWT validation is covered by extensive unit tests, covering validation of all JWT timestamps, validation of the issuers, validation of the issuer's documented supported signing algorithms.
This PR serves as a core, and many enhancements are required for this to be a usable system for users.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [ ] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- Documentation updates for new config entries will be authored.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
- Marking not visible as there's no mechanism to interact with this backend yet.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12261
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
This PR is part of a series (#11311).
Adds support for reading and creating repo-secific access tokens through the API via the `GET /users/{username}/tokens`, `POST /users/{username}/tokens`, and `DELETE /users/{username}/tokens/{id}` APIs.
Validation rules are included to [restrict repo-specific access tokens to specific scopes](https://codeberg.org/forgejo/design/issues/50#issuecomment-11093951).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/11504): <!--number 11504 --><!--line 0 --><!--description cmVhZCwgY3JlYXRlLCAmIGRlbGV0ZSByZXBvLXNwZWNpZmljIGFjY2VzcyB0b2tlbnMgdmlhIEFQSQ==-->read, create, & delete repo-specific access tokens via API<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11504
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
`Option[T]` currently exposes a method `Value()` which is permitted to be called on an option that has a value, and an option that doesn't have a value. This API is awkward because the behaviour if the option doesn't have a value isn't clear to the caller, and, because almost all accesses end up being `.Has()?` then `OK, use .Value()`.
`Get() (bool, T)` is added as a better replacement, which both returns whether the option has a value, and the value if present. Most call-sites are rewritten to this form.
`ValueOrZeroValue()` is a direct replacement that has the same behaviour that `Value()` had, but describes the behaviour if the value is missing.
In addition to the current API being awkward, the core reason for this change is that `Value()` conflicts with the `Value()` function from the `driver.Valuer` interface. If this interface was implemented, it would allow `Option[T]` to be used to represent a nullable field in an xorm bean struct (requires: https://code.forgejo.org/xorm/xorm/pulls/66).
_Note:_ changes are extensive in this PR, but are almost all changes are easy, mechanical transitions from `.Has()` to `.Get()`. All of this work was performed by hand.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11218
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Currently `DeriveKey` is called every time that a secret must be encoded/decoded. Since this function is deterministic, its result can be cached to allow a 250x speedup (the original took less than half a microsecond, so this more of a micro-optimization...).
```
go test -bench=.
goos: linux
goarch: amd64
pkg: forgejo.org/modules/keying
cpu: Intel(R) Core(TM) Ultra 5 125H
BenchmarkExpandPRK-18 2071627 564.2 ns/op
BenchmarkExpandPRKOnce-18 541438192 2.206 ns/op
PASS
ok forgejo.org/modules/keying 2.369s
```
## Other changes
- Since the keys can be constructed once, it simplifies a bit the callsites (`keying.TOTP.Encrypt(...)` instead of `keying.DeriveKey(keying.ContextTOTP).Encrypt(...)`)
- All `Encrypt`/`Decrypt` calls will panic forever if called before `Init` has been called (current it panics as long as `Init` has not been called)
- Calling `Init` twice with different keys will trigger a panic (currently racy)
- Calling `Decrypt` with a short ciphertext does not panic anymore (like when calling with long-enough garbage)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10114
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: oliverpool <git@olivier.pfad.fr>
Co-committed-by: oliverpool <git@olivier.pfad.fr>
Adds a foreign key to the table `forgejo_auth_token` (`AuthorizationToken` in go code). A review has shown that most capabilities around auth tokens seem to be well covered by test automation, but supplemented with manual testing around user deletion as well.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9886
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This patch contains two fixes/enhancements to two functions that were updating all columns of the `access_token` and `repository` table when they were only updating a select few columns. Within Codeberg we saw these two queries quite often when something problematic with the database was going on, likely because of this all columns update pattern. `UpdateAccessToken` is removed and a new function `UpdateLastUsed` was added, for `updateRepoRunsNumbers` we can simply add which columns we want to have updated in that query.
It's likely there are more of such queries, but these were the ones being executed often.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9572
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Otto <otto@codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
TL;DR we can test if transactions are actually working.
Forgejo has many helper functions to test various aspects of the database state, however one aspect it is not able to test is transactions. As this would require that some random SQL query fails to indeed observe that the whole transaction is being rollbacked.
So how do we make a random SQL query fail? Via a fault injector hook, which is always added to xorm (during unittest) and can be enabled on demand to say after how many SQL queries it should start returning a error (fault injecting).
This allows a test to do the following: after two SQL queries lets call it a day and then execute the function that starts a transaction and does a few SQL query. It can then observe that indeed the function was fault injected (`ErrFaultInjected` is returned) and after querying the database it can observe that nothing was changed and thus can conclude the transaction was rollbacked.
---
To demonstrate how the fault injector test helper can be used, lets add a test to a function I really wanted to test but couldn't because the fault injector didn't exist. `NewTwoFactor` was recently made into a transaction (a8c61532d2) and although it would not be catastrophic it would be really bad if records were being inserted if for some reason setting the secret failed.
The test that's added demonstrates that the function uses a transaction and rollbacks correctly.
Weirdly enough the fault injector can be viewed as testing a specification, because it assumes nothing about how the function does it (and you could even design a function that purposely doesn't work but succeeds this test), it merely assumes there's a transaction and within that transaction some SQL queries will be done. However it also needs a certain amount of knowledge about how the function is implemented because the developer needs to tell after how many SQL queries you want to inject a fault and you want to do at a point where there's already a observable change happening in the transaction and not fault inject if the transaction only contains `SELECT` queries.
I'm sure you could design a smart fault injector that can do such guess work (although it sounds like a topic for a PhD thesis) and you could design a helper function that can then guide the fault injector to find every interesting place to do a fault injection and ensure the transaction always falls back; as a first prototype having the programmer tell after how many SQL queries a fault should be injected is sufficient for a lot of the transaction we are going to test in Forgejo.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9314
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
- Currently the options `pubkey` and `twofa` only consider TOTP and GPG keys respectively. Adjust the code to also consider WebAuthn credentials and SSH keys.
- While adding the new unified functions I noticed that certain places also benefited from using these unified functions and took the liberty (where it was either a trivial translation or it was covered under testing) to use the new unified functions.
- Resolvesforgejo/forgejo#7658
- Adds unit and integration tests.
Documentation PR: https://codeberg.org/forgejo/docs/pulls/1166
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7693
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
The only parameter that is ever used is a single directory, make it that only instead of a more complex option structure.
Remove tests.AddFixtures that was the simpler form because it is now redundant.
---
Backporting to v11.0 will help with automated backporting of bug fixes in need of custom made fixtures.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7648
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
## Dropping SSPI auth support
SSPI authentication relied on Microsoft Windows support, removal started in https://codeberg.org/forgejo/forgejo/pulls/5353, because it was broken anyway. We have no knowledge of any users using SSPI authentication. However, if you somehow managed to run Forgejo on Windows, or want to upgrade from a Gitea version which does, please ensure that you do not use SSPI as an authentication mechanism for user accounts. Feel free to reach out if you need assistance.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7148
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Otto Richter <otto@codeberg.org>
Co-committed-by: Otto Richter <otto@codeberg.org>
This commit has a fundamental flaw, in order to syncronize if external
users are still active the commit checks if the refresh token is
accepted by the OAuth provider, if that is not the case it sees that as
the user is disabled and sets the is active field to `false` to signal
that. Because it might be possible (this commit makes this a highly
likelyhood) that the OAuth provider still recognizes this user the
commit introduces code to allow users to re-active themselves via the
oauth flow if they were disabled because of this. However this code
makes no distinction in why the user was disabled and always re-actives
the user.
Thus the reactivation via the OAuth flow allows users to bypass the
manually activation setting (`[service].REGISTER_MANUAL_CONFIRM`) or if
the admin for other reasons disabled the user.
This reverts commit 21fdd28f08.
- Currently the TOTP secrets are stored using the `secrets` module with
as key the MD5 hash of the Secretkey, the `secrets` module uses general
bad practices. This patch migrates the secrets to use the `keying`
module (#5041) which is easier to use and use better practices to store
secrets in databases.
- Migration test added.
- Remove the Forgejo migration databases, and let the gitea migration
databases also run forgejo migration databases. This is required as the
Forgejo migration is now also touching tables that the forgejo migration
didn't create itself.
Add the default value of the purpose field to both the table and the
migration. The table in v9 and v7 backport already have the default
value.
ALTER TABLE `forgejo_auth_token` ADD `purpose` TEXT NOT NULL [] - Cannot add a NOT NULL column with default value NULL
- Add a `purpose` column, this allows the `forgejo_auth_token` table to
be used by other parts of Forgejo, while still enjoying the
no-compromise architecture.
- Remove the 'roll your own crypto' time limited code functions and
migrate them to the `forgejo_auth_token` table. This migration ensures
generated codes can only be used for their purpose and ensure they are
invalidated after their usage by deleting it from the database, this
also should help making auditing of the security code easier, as we're
no longer trying to stuff a lot of data into a HMAC construction.
-Helper functions are rewritten to ensure a safe-by-design approach to
these tokens.
- Add the `forgejo_auth_token` to dbconsistency doctor and add it to the
`deleteUser` function.
- TODO: Add cron job to delete expired authorization tokens.
- Unit and integration tests added.
- Make use of `test.MockVariableValue` to override variables for the
duration of the test.
- Don't needlessly call `onGiteaRun`, its only needed when a HTTP server
needs to be called by the code.
- When `onGiteaRun` is used, make use of the passed parameters, such as
the passed `*testing.T` variable and `*url.URL` (this also avoids
needing to serve the routers in the test code again).
- Use `(*url.URL).JoinPath` to craft new URLs.
- Don't override `setting.AppURL` & `setting.Database.LogSQL` when its
does not affect the test.
- Add empty fixture files for `FederatedUser` & `FederationHost` so they
are truncated and do not persist between tests.
- For WebAuthn Credential level 3, the `backup_eligible` and
`backup_state` flags are checked if they are consistent with the values
given on login. Forgejo never stored this data, so add a database
migration that makes all webauthn credentials 'legacy' and on the next
first use capture the values of `backup_eligible` and `backup_state`.
As suggested in https://github.com/go-webauthn/webauthn/discussions/219#discussioncomment-10429662
- Adds unit tests.
- Add E2E test.
This leverages the existing `sync_external_users` cron job to
synchronize the `IsActive` flag on users who use an OAuth2 provider set
to synchronize. This synchronization is done by checking for expired
access tokens, and using the stored refresh token to request a new
access token. If the response back from the OAuth2 provider is the
`invalid_grant` error code, the user is marked as inactive. However, the
user is able to reactivate their account by logging in the web browser
through their OAuth2 flow.
Also changed to support this is that a linked `ExternalLoginUser` is
always created upon a login or signup via OAuth2.
Ideally, we would also refresh permissions from the configured OAuth
provider (e.g., admin, restricted and group mappings) to match the
implementation of LDAP. However, the OAuth library used for this `goth`,
doesn't seem to support issuing a session via refresh tokens. The
interface provides a [`RefreshToken`
method](https://github.com/markbates/goth/blob/master/provider.go#L20),
but the returned `oauth.Token` doesn't implement the `goth.Session` we
would need to call `FetchUser`. Due to specific implementations, we
would need to build a compatibility function for every provider, since
they cast to concrete types (e.g.
[Azure](https://github.com/markbates/goth/blob/master/providers/azureadv2/azureadv2.go#L132))
---------
Co-authored-by: Kyle D <kdumontnu@gmail.com>
(cherry picked from commit 416c36f3034e228a27258b5a8a15eec4e5e426ba)
Conflicts:
- tests/integration/auth_ldap_test.go
Trivial conflict resolved by manually applying the change.
- routers/web/auth/oauth.go
Technically not a conflict, but the original PR removed the
modules/util import, which in our version, is still in use. Added it
back.
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
A remote user (UserTypeRemoteUser) is a placeholder that can be
promoted to a regular user (UserTypeIndividual). It represents users
that exist somewhere else. Although the UserTypeRemoteUser already
exists in Forgejo, it is neither used or documented.
A new login type / source (Remote) is introduced and set to be the login type
of remote users.
Type UserTypeRemoteUser
LogingType Remote
The association between a remote user and its counterpart in another
environment (for instance another forge) is via the OAuth2 login
source:
LoginName set to the unique identifier relative to the login source
LoginSource set to the identifier of the remote source
For instance when migrating from GitLab.com, a user can be created as
if it was authenticated using GitLab.com as an OAuth2 authentication
source.
When a user authenticates to Forejo from the same authentication
source and the identifier match, the remote user is promoted to a
regular user. For instance if 43 is the ID of the GitLab.com OAuth2
login source, 88 is the ID of the Remote loging source, and 48323
is the identifier of the foo user:
Type UserTypeRemoteUser
LogingType Remote
LoginName 48323
LoginSource 88
Email (empty)
Name foo
Will be promoted to the following when the user foo authenticates to
the Forgejo instance using GitLab.com as an OAuth2 provider. All users
with a LoginType of Remote and a LoginName of 48323 are examined. If
the LoginSource has a provider name that matches the provider name of
GitLab.com (usually just "gitlab"), it is a match and can be promoted.
The email is obtained via the OAuth2 provider and the user set to:
Type UserTypeIndividual
LogingType OAuth2
LoginName 48323
LoginSource 43
Email foo@example.com
Name foo
Note: the Remote login source is an indirection to the actual login
source, i.e. the provider string my be set to a login source that does
not exist yet.
