Handle Qt6/non-desktop installation patching (#210)

* Do not check qmake for android

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Refactoring patching

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: test android_armv7 in 6.0.1

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Patching qt6 android qmake script

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Patching qt6 android qmake script(wip)

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: test android/qt6 on github actions

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Add combination for Qt6 and update README

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Azure: Fix test cases

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Azure: Fix Qt6 test case

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Fix windows case

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Fix windows case

Add debug print

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Fix prefix path for patching

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Improvve log messages for patching

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Check android installation

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Check android installation

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Check android installation

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Updater: update logging message

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Updater: Fix qmake detection on windows

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: run qmake.bat on windows

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: run Qt6 qmake.bat on windows

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: run Qt6 qmake.bat on windows

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Azure: fix mirror test

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: Azure: test Qt6 with Ubuntu 20.04

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
This commit is contained in:
Hiroshi Miura
2021-02-11 10:05:13 +09:00
committed by GitHub
parent ca63ddd771
commit 060a1125f3
9 changed files with 195 additions and 108 deletions

View File

@@ -8,16 +8,19 @@ jobs:
strategy:
matrix:
os: [windows-latest, macOS-latest, ubuntu-latest]
py: [3.6, 3.8]
qtver: [5.9.9, 5.12.8, 5.15.1]
py: [3.6, 3.9]
qtver: [5.9.9, 5.12.8, 5.15.2]
include:
- os: ubuntu-20.04
py: 3.8
qtver: 6.0.0
py: 3.9
qtver: 6.0.1
- os: windows-latest
py: 3.9
qtver: 6.0.1
exclude:
- os: windows-latest
py: 3.6
qtver: 5.15.1
qtver: 5.15.2
steps:
- uses: actions/checkout@v1
with:
@@ -47,6 +50,8 @@ jobs:
args = [qtver, "windows", "desktop", "win64_msvc2019_64"]
elif qtver.startswith('5.14'):
args = [qtver, "windows", "desktop", "win64_msvc2017_64"]
elif qtver.startswith('6'):
args = [qtver, "windows", "desktop", "win64_mingw81"]
else:
args = [qtver, "windows", "desktop", "win64_msvc2015_64"]
elif platform == "macOS-latest":
@@ -55,6 +60,19 @@ jobs:
args = [qtver, "linux", "desktop", "gcc_64"]
command_line.extend(args)
command_line.extend(["--archives", "qtbase", "icu", "qt"])
print("Execute: {}".format(command_line))
try:
res = subprocess.run(command_line, timeout=timeout, check=True)
except subprocess.CalledProcessError as cpe:
exit(cpe.returncode)
assert res.returncode == 0
if qtver.startswith('6'):
if platform == 'ubuntu-20.04':
command_line = ["python", "-m", "aqt", "install", qtver, "linux", "android", "android_armv7"]
elif platform == "macOS-latest":
command_line = ["python", "-m", "aqt", "install", qtver, "mac", "ios", "ios"]
else:
command_line = ["python", "-m", "aqt", "install", qtver, "windows", "android", "android_armv7"]
try:
res = subprocess.run(command_line, timeout=timeout, check=True)
except subprocess.CalledProcessError as cpe:
@@ -63,6 +81,7 @@ jobs:
shell: python
- name: Test qmake -query
run: |
import os
import pathlib
from subprocess import CalledProcessError, PIPE, run
platform = "${{ matrix.os }}"
@@ -72,6 +91,8 @@ jobs:
arch_dir = 'msvc2019_64'
elif qtver.startswith('5.14'):
arch_dir = 'msvc2017_64'
elif qtver.startswith('6'):
arch_dir = 'mingw81_64'
else:
arch_dir = 'msvc2015_64'
elif platform == "macOS-latest":
@@ -88,4 +109,19 @@ jobs:
if line.startswith(b'QT_INSTALL_PREFIX'):
result = line[18:].decode('UTF-8')
assert qt_prefix_path.samefile(result)
print('PREFIX {}'.format(result))
if qtver.startswith('6'):
if platform == "windows-latest" and qtver.startswith('6'):
qmake = os.path.join(qtver, 'android_armv7', 'bin', 'qmake.bat')
else:
qmake = os.path.join(qtver, 'android_armv7', 'bin', 'qmake')
try:
res = run([qmake, "-query"], timeout=15, check=True, stdout=PIPE)
except CalledProcessError as cpe:
exit(cpe.returncode)
assert res.returncode == 0
for line in res.stdout.splitlines():
if line.startswith(b'QT_INSTALL_PREFIX'):
result = line[18:].decode('UTF-8')
print('PREFIX {}'.format(result))
shell: python

View File

@@ -83,7 +83,7 @@ You can also call with ``python -m aqt`` syntax as well as command script ``aqt`
* `win64_mingw53`, `win32_mingw53`,
* `win64_msvc2019_winrt_x64`, `win64_msvc2019_winrt_x86`, `win64_msvc2019_winrt_armv7`
* `win64_msvc2017_winrt_x64`, `win64_msvc2017_winrt_x86`, `win64_msvc2017_winrt_armv7`
* For android and Qt 5.13 or below, choose one of: `android_x86_64`, `android_arm64_v8a`, `android_x86`,
* For android and Qt 6 or Qt 5.13 and below, choose one of: `android_x86_64`, `android_arm64_v8a`, `android_x86`,
`android_armv7`
* You can specify external 7zip command path instead of built-in extractor.
* When specifying `all` for extra modules option `-m` all extra modules are installed.
@@ -139,6 +139,13 @@ Example: Installing Android (armv7) Qt 5.10.2:
aqt install 5.10.2 linux android android_armv7
Example: Installing Android Qt 5.15.2:
.. code-block:: bash
aqt install 5.15.2 linux android android
Example: Install examples, doc and source:
.. code-block:: bash
@@ -148,7 +155,7 @@ Example: Install examples, doc and source:
C:\ aqt src 5.15.0 windows desktop
Example: Install Web Assembly
Example: Install Web Assembly for Qt5
.. code-block:: bash
@@ -178,6 +185,15 @@ Example: Install MinGW on Windows
c:\ set PATH=C:\Qt\Tools\mingw810_64\bin
Example: Install Qt6 for android
.. code-block:: bash
aqt install -O qt 6.1.0 linux desktop
aqt install -O qt 6.1.0 linux android android_armv7
qt/6.1.0/android_armv7/bin/qmake -query
Example: Show help message
.. code-block:: bash

View File

@@ -20,7 +20,6 @@
{"os_name": "windows", "target": "desktop", "arch": "win32_mingw81"},
{"os_name": "windows", "target": "desktop", "arch": "win64_mingw73"},
{"os_name": "windows", "target": "desktop", "arch": "win32_mingw73"},
{"os_name": "windows", "target": "desktop", "arch": "win64_mingw53"},
{"os_name": "windows", "target": "desktop", "arch": "win32_mingw53"},
{"os_name": "windows", "target": "winrt", "arch": "win64_msvc2019_winrt_x64"},
{"os_name": "windows", "target": "winrt", "arch": "win64_msvc2019_winrt_x86"},
@@ -99,7 +98,8 @@
"qtquick3d", "qtquicktimeline", "qtscript", "qtvirtualkeyboard", "qtwebengine", "qtwebglplugin"]},
{"qt_version": "5.15", "modules": ["qtcharts", "qtlottie", "qtnetworkauth", "qtpurchasing", "qtdatavis3d",
"qtquick3d", "qtquicktimeline", "qtscript", "qtvirtualkeyboard", "qtwebengine", "qtwebglplugin"]},
{"qt_version": "6.0", "modules": ["qtnetworkauth"]}
{"qt_version": "6.0", "modules": ["qtwaylandcompositor"]},
{"qt_version": "6.1", "modules": ["qtwaylandcompositor"]}
], "versions": [
"5.5", "5.6", "5.7", "5.8",
"5.9", "5.9.1", "5.9.2", "5.9.3", "5.9.4", "5.9.5", "5.9.6", "5.9.7", "5.9.8", "5.9.9",
@@ -108,5 +108,5 @@
"5.13.0", "5.13.1", "5.13.2",
"5.14.0", "5.14.1", "5.14.2",
"5.15.0", "5.15.1", "5.15.2",
"6.0.0"
"6.0.0", "6.0.1", "6.1.0"
]}]

View File

@@ -82,9 +82,16 @@ class Cli:
return False
def _check_qt_arg_combination(self, qt_version, os_name, target, arch):
if qt_version.startswith('5.15.0') and os_name == 'windows' and target == 'desktop':
if os_name == 'windows' and target == 'desktop':
# check frequent mistakes
if qt_version.startswith('5.15.') or qt_version.startswith('6.'):
if arch in ['win64_msvc2017_64', 'win32_msvc2017', 'win64_mingw73', 'win32_mingw73']:
return False
elif qt_version.startswith('5.9.') or qt_version.startswith('5.10.') or qt_version.startswith('5.11.'):
if arch in ['win64_mingw73', 'win32_mingw73', 'win64_mingw81', 'win32_mingw81']:
return False
elif arch in ['win64_msvc2019_64', 'win32_msvc2019', 'win64_mingw81', 'win32_mingw81']:
return False
for c in self.settings.qt_combinations:
if c['os_name'] == os_name and c['target'] == target and c['arch'] == arch:
return True
@@ -228,7 +235,7 @@ class Cli:
target_config = qt_archives.get_target_config()
self.call_installer(qt_archives, output_dir, sevenzip)
if not nopatch:
finisher(target_config, base_dir, self.logger)
Updater.update(target_config, base_dir, self.logger)
self.logger.info("Finished installation")
self.logger.info("Time elasped: {time:.8f} second".format(time=time.perf_counter() - start_time))
@@ -396,6 +403,7 @@ class Cli:
"\nwindows/desktop: win64_msvc2019_64, win32_msvc2019"
"\n win64_msvc2017_64, win32_msvc2017"
"\n win64_msvc2015_64, win32_msvc2015"
"\n win64_mingw81, win32_mingw81"
"\n win64_mingw73, win32_mingw73"
"\n win32_mingw53"
"\n wasm_32"
@@ -523,43 +531,3 @@ def installer(qt_archive, base_dir, command, response_timeout=30):
raise cpe
os.unlink(archive)
logger.info("Finished installation of {} in {}".format(archive, time.perf_counter() - start_time))
def finisher(target, base_dir, logger):
"""
Make Qt configuration files, qt.conf and qtconfig.pri.
Call updater to update pkgconfig and patch Qt5Core and qmake
"""
qt_version = target.version
arch = target.arch
if arch is None:
arch_dir = ''
elif arch.startswith('win64_mingw'):
arch_dir = arch[6:] + '_64'
elif arch.startswith('win32_mingw'):
arch_dir = arch[6:] + '_32'
elif arch.startswith('win'):
arch_dir = arch[6:]
else:
arch_dir = arch
try:
# prepare qt.conf
with open(os.path.join(base_dir, qt_version, arch_dir, 'bin', 'qt.conf'), 'w') as f:
f.write("[Paths]\n")
f.write("Prefix=..\n")
# update qtconfig.pri only as OpenSource
with open(os.path.join(base_dir, qt_version, arch_dir, 'mkspecs', 'qconfig.pri'), 'r+') as f:
lines = f.readlines()
f.seek(0)
f.truncate()
for line in lines:
if line.startswith('QT_EDITION ='):
line = 'QT_EDITION = OpenSource\n'
if line.startswith('QT_LICHECK ='):
line = 'QT_LICHECK =\n'
f.write(line)
except IOError as e:
raise e
prefix = pathlib.Path(base_dir) / target.version / target.arch
updater = Updater(prefix, logger)
updater.qtpatch(target)

View File

@@ -31,7 +31,6 @@ class Updater:
self.prefix = prefix
self.qmake_path = None
self.qconfigs = {}
self._detect_qmake(prefix)
def _patch_qtcore(self, lib_dir, components, encoding):
for component in components:
@@ -65,48 +64,114 @@ class Updater:
file.write_text(data, "UTF-8")
os.chmod(str(file), st.st_mode)
def _detect_qmake(self, prefix):
""" detect Qt configurations from qmake
"""
for qmake_path in [prefix.joinpath('bin', 'qmake'), prefix.joinpath('bin', 'qmake.exe')]:
def _detect_qmake(self) -> bool:
"""detect Qt configurations from qmake."""
for qmake_path in [self.prefix.joinpath('bin', 'qmake'), self.prefix.joinpath('bin', 'qmake.exe')]:
if not qmake_path.exists():
return
continue
try:
result = subprocess.run([str(qmake_path), '-query'], stdout=subprocess.PIPE)
except subprocess.SubprocessError:
return
else:
return False
if result.returncode == 0:
self.qmake_path = qmake_path
for line in result.stdout.splitlines():
vals = line.decode('UTF-8').split(':')
self.qconfigs[vals[0]] = vals[1]
return True
return False
def _versiontuple(self, v: str):
return tuple(map(int, (v.split("."))))
def qtpatch(self, target):
""" patch works """
if target.os_name == 'linux':
self.logger.info("Patching pkgconfig configurations")
self._patch_pkgconfig(self.prefix.joinpath("lib", "pkgconfig"))
def patch_qmake(self):
"""Patch to qmake binary"""
if self._detect_qmake():
self.logger.info("Patching {}".format(str(self.qmake_path)))
self._patch_binfile(self.qmake_path, key=b"qt_prfxpath=", newpath=bytes(str(self.prefix), 'UTF-8'))
if target.arch not in ['ios', 'android', 'wasm_32', 'android_x86_64', 'android_arm64_v8a', 'android_x86',
'android_armv7']:
def patch_qmake_script(self, base_dir, qt_version, os_name):
if os_name == 'linux':
self.logger.info("Patching {}/bin/qmake".format(self.prefix))
self._patch_textfile(self.prefix / 'bin' / 'qmake',
'/home/qt/work/install/bin',
'{}/{}/{}/bin'.format(base_dir, qt_version, 'gcc_64'))
elif os_name == 'mac':
self.logger.info("Patching {}/bin/qmake".format(self.prefix))
self._patch_textfile(self.prefix / 'bin' / 'qmake',
'/Users/qt/work/install/bin',
'{}/{}/{}/bin'.format(base_dir, qt_version, 'clang_64'))
elif os_name == 'windows':
self.logger.info("Patching {}/bin/qmake.bat".format(self.prefix))
self._patch_textfile(self.prefix / 'bin' / 'qmake.bat',
'/Users/qt/work/install/bin',
'{}\\{}\\{}\\bin'.format(base_dir, qt_version, 'mingw81_64'))
else:
pass
def qtpatch(self, target):
""" patch to QtCore"""
if target.os_name == 'mac':
self.logger.info("Patching QtCore")
self._patch_qtcore(self.prefix.joinpath("lib", "QtCore.framework"), ["QtCore", "QtCore_debug"], "UTF-8")
elif target.os_name == 'linux':
self.logger.info("Patching libQt(5|6)Core")
self._patch_pkgconfig(self.prefix.joinpath("lib", "pkgconfig"))
self._patch_qtcore(self.prefix.joinpath("lib"), ["libQt5Core.so", "libQt6Core.so"], "UTF-8")
elif target.os_name == 'windows':
self.logger.info("Patching Qt(5|6)Core.dll")
self._patch_qtcore(self.prefix.joinpath("bin"), ["Qt5Cored.dll", "Qt5Core.dll", "Qt6Core.dll",
"Qt6Cored.dll"], "UTF-8")
else:
# no need to patch Qt5Core
pass
if self.qmake_path is not None:
self.logger.info("Patching qmake")
self._patch_binfile(self.qmake_path, key=b"qt_prfxpath=", newpath=bytes(str(self.prefix), 'UTF-8'))
def make_qtconf(self, base_dir, qt_version, arch_dir):
"""Prepare qt.conf"""
with open(os.path.join(base_dir, qt_version, arch_dir, 'bin', 'qt.conf'), 'w') as f:
f.write("[Paths]\n")
f.write("Prefix=..\n")
def set_license(self, base_dir, qt_version, arch_dir):
"""Update qtconfig.pri as OpenSource"""
with open(os.path.join(base_dir, qt_version, arch_dir, 'mkspecs', 'qconfig.pri'), 'r+') as f:
lines = f.readlines()
f.seek(0)
f.truncate()
for line in lines:
if line.startswith('QT_EDITION ='):
line = 'QT_EDITION = OpenSource\n'
if line.startswith('QT_LICHECK ='):
line = 'QT_LICHECK =\n'
f.write(line)
@classmethod
def update(cls, target, base_dir, logger):
"""
Make Qt configuration files, qt.conf and qtconfig.pri.
And update pkgconfig and patch Qt5Core and qmake
"""
qt_version = target.version
arch = target.arch
if arch is None:
arch_dir = ''
elif arch.startswith('win64_mingw'):
arch_dir = arch[6:] + '_64'
elif arch.startswith('win32_mingw'):
arch_dir = arch[6:] + '_32'
elif arch.startswith('win'):
arch_dir = arch[6:]
else:
arch_dir = arch
try:
prefix = pathlib.Path(base_dir) / target.version / arch_dir
updater = Updater(prefix, logger)
updater.set_license(base_dir, qt_version, arch_dir)
if target.arch not in ['ios', 'android', 'wasm_32', 'android_x86_64', 'android_arm64_v8a', 'android_x86',
'android_armv7']: # desktop version
updater.make_qtconf(base_dir, qt_version, arch_dir)
updater.qtpatch(target)
updater.patch_qmake()
elif qt_version.startswith('5.'): # qt5 non-desktop
updater.patch_qmake()
else: # qt6 non-desktop
updater.patch_qmake_script(base_dir, qt_version, target.os_name)
except IOError as e:
raise e

View File

@@ -62,14 +62,14 @@ jobs:
displayName: Linux (Specific Mirror)
variables:
PYTHON_VERSION: '3.8'
QT_VERSION: 5.13.1
QT_VERSION: 6.1.0
HOST: linux
TARGET: android
ARCH: android_armv7
ARCHDIR: android_armv7
TARGET: desktop
ARCH: gcc_64
ARCHDIR: gcc_64
QT_BASE_MIRROR: http://mirrors.ocf.berkeley.edu/qt/
pool:
vmImage: 'ubuntu-18.04'
vmImage: 'ubuntu-20.04'
steps:
- template: ci/steps.yml

View File

@@ -56,8 +56,8 @@ for qt_version in qt_versions:
# Mac iOS, android
mac_build_jobs.extend(
[
BuildJob('5.13.2', 'mac', 'ios', 'ios', 'ios'),
BuildJob('5.14.1', 'mac', 'android', 'android', 'android')
BuildJob('5.15.2', 'mac', 'ios', 'ios', 'ios'),
BuildJob('5.14.2', 'mac', 'android', 'android', 'android')
]
)
@@ -97,8 +97,10 @@ windows_build_jobs.append(
)
# android
linux_build_jobs.append(
BuildJob('5.14.1', 'linux', 'android', 'android', 'android')
linux_build_jobs.extend(
[
BuildJob('5.14.2', 'linux', 'android', 'android', 'android'),
]
)
matrices = {}

View File

@@ -111,18 +111,18 @@ steps:
##----------------------------------------------------
# for Android on linux
- script: |
wget https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip
unzip android-ndk-r20b-linux-x86_64.zip
wget https://dl.google.com/android/repository/android-ndk-r21e-linux-x86_64.zip
unzip android-ndk-r21e-linux-x86_64.zip
condition: and(eq(variables['TARGET'], 'android'), eq(variables['Agent.OS'], 'Linux'))
displayName: Download and extract Android NDK
# for Android on mac
- script: |
wget https://dl.google.com/android/repository/android-ndk-r20b-darwin-x86_64.zip
unzip android-ndk-r20b-darwin-x86_64.zip
wget https://dl.google.com/android/repository/android-ndk-r21e-darwin-x86_64.zip
unzip android-ndk-r21e-darwin-x86_64.zip
condition: and(eq(variables['TARGET'], 'android'), eq(variables['Agent.OS'], 'Darwin'))
displayName: Download and extract Android NDK
- bash: |
export ANDROID_NDK_ROOT=$(Build.SourcesDirectory)/android-ndk-r20b
export ANDROID_NDK_ROOT=$(Build.SourcesDirectory)/android-ndk-r21e
mkdir $(Build.BinariesDirectory)/tests
(cd $(Build.BinariesDirectory)/tests; 7zr x $(Build.SourcesDirectory)/ci/accelbubble.7z)
export PATH=$(Build.BinariesDirectory)/Qt/$(QT_VERSION)/$(ARCHDIR)/bin:$PATH

View File

@@ -99,11 +99,11 @@ windows
| | - 5.10.1, 5.10.0 |
| win32_msvc2015 | - 5.9.9, 5.9.8, 5.9.7, 5.9.6, 5.9.5 |
+--------------------+---------------------------------------------+
| win32_mingw81 | - 6.0.1, 6.0.0 |
| | - 5.15.2, 5.15.1, 5.15.0 |
| - win64_mingw81 | - 6.0.1, 6.0.0 |
| - win32_mingw81 | - 5.15.2, 5.15.1, 5.15.0 |
+--------------------+---------------------------------------------+
| win32_mingw73 | - 5.14.2, 5.14.1, 5.14.0 |
| | - 5.13.2 5.13.1, 5.13.0 |
| - win64_mingw73 | - 5.14.2, 5.14.1, 5.14.0 |
| - win32_mingw73 | - 5.13.2 5.13.1, 5.13.0 |
| | - 5.12.9, 5.12.8, 5.12.7, 5.12.6, 5.12.5, |
| | 5.12.4, 5.12.3, 5.12.2, 5.12.1, 5.12.0 |
+--------------------+---------------------------------------------+