* Add authentication flags for list-qt-commercial, add tests for coverage
* Add dry run
* Make tests really use auth since secrets have been added, fix some indents
* Fix auth issue, rename user 'email, and password 'pw'
* Fix modules param type
* Update commands names
* Rewrite WASM support
* Add WASM tests to CI, update CI to test more the latest versions, add auto EMSDK version detection function
* Fix some mistakes, typos, moved emsdk version function into BuildJob
* Fix issue related to extensions interfering with wasm on 6.8+
* Fix tests
* Remove dep on Version in CI
* Remove safety before patch
* handle cases where extensions don't exist.
for example with windows desktop 6.8.1 win64_msvc2022_arm64_cross_compiled
both qtwebengine and qtpdf don't exist.
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* for --long-modules assume extension doesn't exist on download error.
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* for --modules assume extension doesn't exist for download failures.
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* reformat with black
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* fix flake8 regression that doesn't occur locally.
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* Fix autodesktop by also updating the OS when searching for a valid desktop version to download
* Fix extension issue, reduce the possible retry for getting extensions to prevent server spam
* Fix CI asking for msvc2019 on 6.8+ but its no longer supported
* Make CI use C++20 and MSVC2022
* Fix linux build
* Update runners to windows-2022
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* Fix patching
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* Add back the semantic version changes to prevent crashes, add tests for it
* Update checks
* Cast 'https://mirrors.ustc.edu.cn' to the shadow realm
* Again
* Update settings.ini
* Update settings.ini
* Update settings.ini
* Remove one_rep on silent
* Update settings.ini
* Restore master settings, remove hash check
* ci: Use specific mirror
Attempt to work around download errors in Azure due to Qt's official download site often redirecting to mirrors to which the network connection is unstable
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* Re enable hash checking
* Treat read timeout error during download as connection error
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
* Add test for modules in WASM with autodesktop
* Fix format
* Fix test
* Make '--autodesktop' trigger its own install process, and test it
* Fix older autodesktop tests
* Add mock update files for 680 wasm, add test for wasm 680 autodesktop
* Passes the additional tests
* Fix format
* Improve coverage, fix format
* Fix tests and improve logging or install
* Fix format
* Fix regression in other tests
* Use flavor
* Fix line len
* Fix codeql
* Fix list-qt for WASM arch on 6.5.x and 6.6.x, restore to original download URL
* Fix test error
* Revert ci settings URL as it is never used by clients, only in CI
* Add comment for clarity in ci/settings.ini
---------
Signed-off-by: Alexandre 'Kidev' Poumaroux <1204936+Kidev@users.noreply.github.com>
Co-authored-by: tsteven4 <13596209+tsteven4@users.noreply.github.com>
Co-authored-by: J.D. Purcell <jdpurcell@gmail.com>
The directory structure of the Qt libraries has changed enough that
the Qt5 example project we are using for WASM cannot be built using
Qt6. This change adds an example project appropriate for use with
Qt6, and adds build jobs for Mac and Linux that build it too,
to prove that we can build the same code on all 3 platforms.
* Add tests for proper patching of Qt6.2.2-ios
* Add a build target for ios installations
* Add test coverage for mobile patching on linux
This change folds in some refactoring as well, in an attempt to reduce
repetitive logic, and make sure that this logic is used everywhere appropriate.
This change saves PowerShell modules used in the Azure Pipelines to a
cache, and loads them when required. This is intended to prevent random
failures when the PowerShell Gallery fails to provide the requested
modules. These failures occur at random, possibly as a result of a
backpressure mechanism.
This change will also speed up all Windows Azure Pipelines jobs, since
downloads from the PowerShell Gallery will no longer be necessary.
This change updates the usage of `install-src`, `install-doc`, and
`install-example` to omit the optional `TARGET` parameter.
It also changes the usage of `--archives`, so that the archives for
the modules are not be specified.
This also adds customizable checks to each build job, so that specific
requirements can be made on an individual basis.
The three `install-src/doc/example` commands now have specific checks
that will cause the build job to fail if the requested modules and
archives have not been installed.
This updates CI and unit tests to monitor behavior of
`install-src/doc/example`, now that the `target` parameter has been
deprecated. This makes sure that the `target` parameter is optional,
and that the CLI warns the user that `target` is deprecated when a user
uses it.
[CI tests]:
This adds a test to `install-src/doc/example` Azure Pipeline jobs to
ensure that the desired files are installed, using `ls`.
This also changes some of the archives installed to `qtlottie`, which
is a particularly small file.
This also adds a build job for `install-example`.
[Unit tests]:
This adds tests for `install-src`, with and without the `target`
parameter.
It also fixes existing tests for `install-doc`, to account for
changes to the `target` parameter.
This commit squashes some existing commits, all related to
the Azure Pipeline CI builds.
WHAT:
1. Use `windows-2019` image in CI
2. Switch Qt download mirror to `https`
3. Replace MSVC-2017 builds: MSVC-2017 is not present in `windows-2019`
4. Fix broken mingw builds
WHY:
As you can see in a recent Azure Pipelines build, the Windows Server
2016 image is now deprecated:
https://dev.azure.com/miurahr/github/_build/results?buildId=4687&view=logs&j=c4497b2b-649a-591e-539b-0d716883e33e
The newer `vmImage: 'windows-2022'` would work too, see
https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml
MINGW:
The existing mingw builds were a little bit broken because the `Path`
environment variable did not include the mingw toolchain installed with
`aqt install-tool`, so the Chocolatey mingw was being used.
This change fixes that by adding parameters to BuildJob that are
specific to each mingw toolchain and fixing the code that sets the Path
environment variable in Powershell.
This change adds the ability to list all the 'tool variant names'
alongside the version and release date of the tool.
When using the `aqt list tools` feature, I often find that I need more
information than the tool provides. I often need to know the release
date and the version for each tool, and I can't get that information
without looking up the Updates.xml file. This feature enables me to skip
that part.
* Add html scraping capability to helper.py
The function scrape_html_for_folders() reads an html file at
`https://download.qt.io/online/qtsdkrepository/<os>/<target>/` and
extracts a list of all the folders within that directory.
It separates the folders by category, so that 'tools' are not
mixed in with 'qt5' or 'qt6'.
This commit also adds a function that prints the available folders,
allowing the caller to filter the output by category.
* Add interface for accessing scrape_html_for_folders
I'm sure this isn't exactly what is needed, and that this functionality
should probably be folded into `aqt list`, but it does demonstrate that
aqt can list all the tools available at downloads.qt.io by scraping html.
This allows you to type `aqt list2 tools linux desktop`, and get a list
of all the tools on the server.
There are more options; type `aqt list2 -h` to see them all.
* Add failing tests for `aqt list`
* Add passing implementation of `aqt list`
* Remove old implementation of `aqt list`
This removes `Cli.run_list_old`, as well as any commented-out code in
installer.py that was related to the old implementation of `run_list`.
This also restores some code that was commented out erroneously.
* Fix misuse of semantic_version
In an earlier commit, I used the parameter `partial=True` for the
preview builds of Qt, since they do not include a patch number. I did
not recognize this as a misuse of semantic_version until I ran the tox
linter. `partial` is meant for 'partial specification', as in, 'this
program requires version 6.2, and the patch is unimportant`.
This change removes the `partial=True` parameter, adds a `0` for the
patch number, and uses a format string to print preview versions without
the patch.
* Use semantic version in interfaces instead of str
This updates all functions that accept a Qt version as a string, so that
they now accept a semantic version object instead. This forces the
caller to do some input validation, and moves the responsibility of
string-to-version conversions out of functions where it does not belong.
* Improve `aqt list` interface as discussed
This adds several new options to `aqt list`, and renames the ambiguous
`packages` to `modules`. New options include the ability to list
targets, extensions, and architectures for a particular (host, target,
Qt version) tuple. The goal is to provide sufficient information to the
user, such that the user can use all the other features of aqt without
having to look them up somewhere else.
* Cleanup `targets` dictionary
* Add `list architectures` feature
* Add tests for `list architectures` feature
* Add documentation for new list feature.
* Fix url bugs related to BASE_URL change
* Add stub for `list --extensions`
* Implement `list --extensions`
* Cleanup imports
* Add basic tests of `aqt list` to azure pipeline
* Fix failed merge with upstream
* Simplify string-to-version code
An earlier commit introduced `cli_2_semantic_version`, which calls
`get_semantic_version_with_dots`, which will never be called by any
other code. I think that this should really just be one function, and
splitting it in two is just needlessly complex.
* Revert dd41e653 - changes to azure-pipelines.yml
* CI: change test case for list command
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
* Update MANIFEST
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
* Fix errors reported by flake8
This commit fixes some unused imports, missing imports, and formatting
errors.
* Run isort
* Add three more build jobs to azure pipelines
This adds three more combinations of `qt version/host/target` to the
`aqt list` command tests on the azure pipelines. This is pretty far from
exhaustive, but I think it's worth it to add these: they execute
slightly different code paths than the standard `linux/desktop`
variants. I don't want to add a whole lot more than this; `aqt list` is
slower than I'd like it to be, and I don't want to waste CI time.
* Fix bug where empty extensions returns success
Prior to this commit, if you ran `aqt list` to list possible extensions,
it could print a blank line and return 0. This is because the
get_extensions_for_version function was accepting empty strings as a
valid extension, and returning a list consisting of one empty string.
This commit filters out those empty strings.
* Attempt to fix ci bug when listing android exts
Android extensions for Qt6 follow different rules than Qt5; this
attempts to account for that.
* remove invalid ArchiveId objects from tests
* Add author
* Use Python for test logic instead of bash
* Remove 'build accelbubble' step from 'list' test
This commit attempts to separate the 'build accelbubble example' test
from the 'aqt list' test.
In this case, the test for `aqt list qt5 linux android` was running the
"build accelbubble example" test in addition to the intended test. If
the "build accelbubble example" test fails, then it looks like the `aqt
list` test failed, until you look to see why it failed.
* Remove dead code
* Accept `latest` keyword instead of qt version
This commit modifies the interface of `aqt list`, so that the flags
`--modules`, `--extensions`, and `--arch` will accept either a Qt
version or the word `latest`. If the user specifies a version of Qt,
the command works as before. If the user specifies `--modules latest`,
`aqt list` will determine the latest version of Qt, given the filters
specified, and return the modules for that version.
This commit removes the flag `--latest-modules`, because that flag is no
longer necessary. The `latest` keyword serves the same function, and it
works for the `--extensions`, and `--arch` flags as well. This prevents
the need for flags such as `--latest-extensions` and `--latest-arch`,
which otherwise would have been necessary to provide this functionality
to the other list features.
* remove unused import
* Update azure pipeline to reflect `latest` feature
* WIP refactor helper methods into ListCommand
* Refactor helper methods into ListCommand
This PR adds many helper methods to helper.py that are closely related
to each other, and which are only useful to the ListCommand object. This
commit moves the majority of these into the ListCommand object, so that
they don't confuse future readers who are looking for something else.
* Fix imports after merge
* Fix unused import
* Add test and impl for list tool modules
This adds code that parses an 'Updates.xml' file for the names of tool
modules, which a user can use to install tools. This does not have any
cli-facing code yet, since the current interface does not make sense for
this purpose.
* Add CLI command to list tool variant names
This exposes `ListCommand.fetch_tool_modules` to the CLI.
Accordingly, it adds a test to the Azure Pipelines for every `aqt list`
job, and it adds documentation to docs/cli.rst.
Since this feature is meant to print values you can feed into the `aqt
tool` command, the documentation for the `aqt tool` command was changed
as well.
* Use monkeypatch for http_fetcher
This removes the http_fetcher parameter for ListCommand in favor of
monkeypatching.
* Remove dataclasses
Dataclasses were new in Python 3.7, and don't exist in the minimum
supported Python 3.6.
* black
* Fix ci test for `aqt list tools --tool`
* Add default timeout to request_http_with_failover
* fix monkeypatched http fetcher
* Fix 'aqt list tools' command on Azure Pipeline
Currently, the `tools_qt3dstudio_runtime_240` appears to be the only
tool available on all host/target combinations, so we will check that
one in Azure Pipelines.
* Move helper classes specific to ListCommand into ListCommand
* Fix use of new global Settings object
* Fix sphinx docstring syntax
* Swap `request_http_with_failover` for `getUrl`
* Add additional follow-up suggestion on error
* Fix exceptions import
* Prevent repeated 'suggested follow up' actions
This change uses exclusive branches to prevent `aqt list` from repeating
the same/similar recommended actions.
* Fix bugs related to updated getUrl error messages
* run black
* docs: add auto section link label extension
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
* docs: add more list command examples
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
* Fix typo in docs
* remove duplicate `semantic_version`
* Add expected outputs and arch example to docs
This change adds some expected output to the documentation. For any
user who is confused about what the documentation means when it
mentions `tools` and `modules`, and is unsure which is which, this
change should help that user figure that out.
This also adds an example for listing architectures.
* Remove `logger` from `getUrl` parameter list
* helper: getLogger("aqt.helper") in helper functions
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
* Use `pretty_print` instead of `str()` for output
* test_list.py use pretty_print instead of str
* fix Versions constructor
* black
Co-authored-by: Hiroshi Miura <miurahr@linux.com>
