For the previous code with the Page attribute present in
ListCursorOptions for page 1, github would not return an "After" cursor,
such that the request for page 2 would request what effectively is the
content of page 1 a second time.
This would lead to an attempt to insert the same issues twice.
Note that this is not the only reason why this can happen with the
current code base.
We fix this particular issue by not using the Page attribute so github
does return an "After" cursor.
Fixes#10794
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10798
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Nils Goroll <nils.goroll@uplex.de>
Co-committed-by: Nils Goroll <nils.goroll@uplex.de>
Related to https://codeberg.org/Codeberg/Community/issues/1944
* Allowed the githubdownloaderv3 to know whether issues and, or PRs are requested to migrate
* Used this information to decide to filter for "/pulls/" or "/issues"
* Or not to filter at all if issues == true && prs == true
* Added isolated test for the downloader and for the uploader
* Created a new test_repo in github.com/forgejo and set it up properly together with @Gusted
* Updated github_downloader_test with the new URLs and test data from the repo
* Recorded the API calls for local testing
* Added a minimal gitbucket test (which uses the github downloader under the hood)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8892
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: patdyn <patdyn@noreply.codeberg.org>
Co-committed-by: patdyn <patdyn@noreply.codeberg.org>
Add support for migrating Pagure repositories (including metadata) to Forgejo via the migration tool.
One 'discrepancy' with migration from other forges is how privates issues are handled, they are migrated when a API token is set for the migration, in that case users are advised to set their repositories visibility to private to avoid leaking such private issues.
Co-authored-by: Akashdeep Dhar <akashdeep.dhar@gmail.com>
Co-authored-by: Ryan Lerch <rlerch@redhat.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8513
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Akashdeep Dhar <akashdeep.dhar@gmail.com>
- When migrating a pull requests from a Gitea or Forgejo instance, check if the pull request was created via the AGit flow and transfer that bit of information to the migrated pull request.
- Expose this bit of information as the `flow` field for the pull request.
- We have to do a horrible Go hack with Gitea's [go-sdk](gitea.com/gitea/go-sdk) to list all pull requests while being able to decode it to a struct that contains the new `Flow` field. The library does not allow you to do this out of the box, so we have to use `go:linkname` to access the private method that allows us to do this. This in turn means we have to do some boilerplate code that the library otherwise would do for us. The better option would be forking, but that would be a hassle of keeping the library in sync.
- Resolvesforgejo/forgejo#5848
- Unit test added.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7421
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
* cleanup: remove not used properties
* feat: implement migration of website field from gogs
* feat: implement dumping and restoring website field
* feat: implement migration of website field from gitea
* feat: implement migration of homepage/website field from github
* feat: implement website properties for repository migration
Gogs migration is untested for now.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6474
Reviewed-by: Otto <otto@codeberg.org>
Co-authored-by: ThomasBoom89 <thomasboom89@noreply.codeberg.org>
Co-committed-by: ThomasBoom89 <thomasboom89@noreply.codeberg.org>
See also: [GITEA] Enable mocked HTTP responses for GitLab migration test
make GITLAB_READ_TOKEN=XXXX GO_TEST_PACKAGES='code.gitea.io/gitea/services/migrations/... -test.v' FLAGS= 'test#TestGitlabDownloadRepo'
to update the captured GitLab HTTP requests used for offline testing.
Closes#1789.
The bug was due to the fact that GitLab does not guarantee that issue numbers are created sequentially: some identifiers can be skipped. Therefore, the new pull requests numbers should not be offset by the number of issues, but by the maximum issue number.
See for instance https://gitlab.com/troyengel/archbuild/-/issues/?sort=created_date&state=all&first_page_size=20, where there is only a singe issue with number "2".
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1790
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
Co-committed-by: Antonin Delpeuch <antonin@delpeuch.eu>
(cherry picked from commit 2c185c39fe600041701d5f59cb1076a788815cb4)
(cherry picked from commit 8f68dc4c9c2f0acab55d59a496b0f141befad969)
(cherry picked from commit 7e932b7fca1b119e7cc646183c383ba51a5f1d14)
(cherry picked from commit 6bbe75ecf8ac502bd42ff5765e6e7733f290a54e)
(cherry picked from commit b18c2e8d658c3311e0a299696bd1b6612c52ef13)
Conflicts:
services/migrations/gitlab.go
https://codeberg.org/forgejo/forgejo/pulls/2075
(cherry picked from commit abc129c762b3c1a992ad5c67adf62d8336eadbbe)
(cherry picked from commit 28884fac10c455a9f40bebd961fca40afd4a749e)
(cherry picked from commit 5f528dd85fa6705c60d15bce616a46f00df1b85b)
(cherry picked from commit cb9b8a31b25b27fa5e386b20f02e532bbf3462a2)
(cherry picked from commit 97f02df163e2b4e23b82e23e5ef57a586b17f021)
(cherry picked from commit 4611e10e6acef9185e1f8c84e63d1fe95fcf6436)
Fix gitlab migration unit test
Closes#1837.
The differences in dates can be explained by commit e19b9653ea, which
changed the order in which "created_date" and "updated_date" are
considered.
(cherry picked from commit b0bba20aa44e30ef0296b89f336d426224d73a16)
Mock HTTP requests in GitLab migration test
This introduces a new utility which can be added to other tests
making HTTP calls to a live service, to cache the responses of this
service in the repository.
(cherry picked from commit 52053b138948bd74c7eb88c0796c2e18f4247f3c)
Enable mocked HTTP responses for GitLab migration test
(cherry picked from commit 19cefc4de24b935a6a5c92be8360301f196f3aa5)
Simplify HTTP mocking utility in unit tests
Follow-up to https://codeberg.org/forgejo/forgejo/pulls/1841
(cherry picked from commit ca517c8bb4bf97f061b8b19fd3303d734f46660c)
(cherry picked from commit b227e0dd6bdf2dc3e8679443fc538fbce4b3bcf5)
(cherry picked from commit 6cc9d06556cda6c952a0542284fbe504114971ce)
(cherry picked from commit f0746e648dc30510d655b8a3b821199b2638800f)
(cherry picked from commit 414193341b8493723c16694789cbc08dc80b9ce5)
(cherry picked from commit 6e93df3bbb6c589502afc9dc74a7ae1a7c0f7da8)
(cherry picked from commit db0dbab5527c9f1783fd0eddb057c2d91cbb67e4)
(cherry picked from commit 8f9c9c63fbd3f266bb29d38791e83dc369cc1350)
(cherry picked from commit e74e26203095b675ccedbc2e166faed59369d467)
(cherry picked from commit 2e0933edcfa102b578fb3c2500f9e6af9e5ba1c7)
(cherry picked from commit 65060c69616631221d3dd9ef8b48fbcb007ad0c6)
