Commit Graph

83 Commits

Author SHA1 Message Date
Hiroshi Miura
a43a26b7c6 Introduce timeout option
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-21 21:06:44 +09:00
Hiroshi Miura
3582152996 Merge pull request #196 from miurahr/topic-fallback
Select fallback mirror site when main site connection failed
2021-01-21 12:11:29 +09:00
Hiroshi Miura
5d71cf4a38 Fix tools installation to use fallback url
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-21 11:15:51 +09:00
Hiroshi Miura
7fd019e83b Fix for checks and tests
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-21 10:09:44 +09:00
Hiroshi Miura
c6f95a7d58 Select fallbacks from known list
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-21 10:04:14 +09:00
Hiroshi Miura
88b6cc6c81 Support --fallback mirror option 2021-01-21 08:21:19 +09:00
Vadim Peretokin
864481aa0d Fix up grammar 2020-12-14 08:34:22 +01:00
Hiroshi Miura
51fe856d1c Refactor: drop settings.py
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-22 15:25:22 +09:00
Hiroshi Miura
df8c4a17ae Support partial source installation
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-22 09:52:58 +09:00
Hiroshi Miura
d8b0ae647c Refactoring
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 22:35:37 +09:00
Hiroshi Miura
8bb70f6f37 Refactoring modules
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 21:17:24 +09:00
Hiroshi Miura
780d9f2ede PEP8
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 18:13:50 +09:00
Hiroshi Miura
6f5cdd2563 Finisher called last
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 17:59:59 +09:00
Hiroshi Miura
da85e50906 multiprocess installation
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 17:43:51 +09:00
Hiroshi Miura
0737f7a810 Fix to catch extraction error
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-08-01 21:02:52 +09:00
Hiroshi Miura
cab78ff8fb Refactoring
- Refactoring archives classes to use inhert functions
- define installer.finalize() and call it explicit from cli.py

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-31 14:24:53 +09:00
Hiroshi Miura
72c4589e7d Patch qmake as finalize process (#130)
* Patch qmake hard coded path with installed prefix(#100)
* AP: Update tests
 - Test android target with 5.14.x
 - Test ios target on mac
* Does not patch mac framework when android, ios and wasm
* Update changelog
* GHA: Update to run qmake to check patch
  - Check qmake works well if patched.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-21 21:56:04 +09:00
Hiroshi Miura
fcb5b45631 CLI: Add --archives option
when specify '--archive qtbase qttools' then
aqt installs these subset of installation.

There is no gurantee it works, and it is an advanced
option.

Implemented #126

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-19 12:14:25 +09:00
Hiroshi Miura
bdb1156351 py7zr depndency bamp up to v0.7.0 (#125)
* Configure all dependency on setup.cfg
* depend on py7zr 0.7

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-09 23:26:33 +09:00
Hiroshi Miura
188b32de8f Fix to use concurrency settings
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-07 13:46:13 +09:00
Hiroshi Miura
caf804f3ae Limit retries with back-off factor
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-06 08:22:33 +09:00
Hiroshi Miura
3b06814601 Refactoring altlink helper (#109)
* Refactoring altlink helper

Also fix a bug not accept a header context-type when
got charset parameter such as
application/meta4+xml; charset=utf-8

* Add test and accept text/plain mime type

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-05 23:00:31 +09:00
Hiroshi Miura
beb755219a Check redirect location before get metalink data
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-04 16:30:16 +09:00
Hiroshi Miura
79b5a297b5 Check version number and improve error messages (#102)
* Check Qt versions and report 404 error

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-01 11:35:39 +09:00
Hiroshi Miura
36238fc007 isort
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-28 08:56:03 +09:00
Kyle Altendorf
0d929826d4 no more os.chdir() 2020-02-26 00:13:39 -05:00
Hiroshi Miura
8b983d0a65 Work around a bug of py7zr may change current working directory
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:32:58 +09:00
Hiroshi Miura
705920b1cc Revert "Use written path as pathlib.Path"
This reverts commit 134ec36dc6.
2020-02-26 10:30:37 +09:00
Hiroshi Miura
134ec36dc6 Use written path as pathlib.Path
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:13:42 +09:00
Hiroshi Miura
ebead77b65 add fd.flush() to writing files
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:05:32 +09:00
Hiroshi Miura
42078a2834 Pass opened file object to extractor
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 09:47:42 +09:00
Hiroshi Miura
3d945ba9f6 Refactoring threading of installation
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 08:31:15 +09:00
Hiroshi Miura
58d8340959 Use concurrent.futures for threading as same as v0.7.*
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-24 10:05:53 +09:00
Hiroshi Miura
81e55518d5 Fix installation logics
- Fix condition to start extraction which leads FileNotFound error.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 15:43:23 +09:00
Hiroshi Miura
25623901e5 Fix extraction check
- An order of list of completed task become as same
  both download and extract.
  This will fix a FileNotFound error.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 14:19:41 +09:00
Hiroshi Miura
6dd63186d8 Get exception on each threads
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 11:57:28 +09:00
Hiroshi Miura
527ce3fd4f Test with py7zr 0.6.*
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-22 09:59:46 +09:00
Hiroshi Miura
4b18f6520d Limit extraction thread number to 4
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 16:08:09 +09:00
Hiroshi Miura
9f15361ace Use only threading for processing
- Now limiting download on fixed concurrency with threading.
- Extraction is unlimited threading, but extraction external process or
  library may run on multiprocessing.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 16:03:12 +09:00
Hiroshi Miura
67e6e1c0cb use Popen for lauching external 7zip command
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 08:46:50 +09:00
Hiroshi Miura
bbb39c0847 Use concurrent.futures for concurrency with thread and process (#87)
Use concurrent.futures for multiprocessing and other improvements
- combination with thread pool for downloading and  process pool for extraction.
- Iterate all download and start extraction.
- Start extraction when download completed
- Add download completion log
- log a time  when extraction done
- Show elasped time for installation
- Meature elasped time
  * use perf_counter() for a total elapsed time.
  * use process_time() for individual extractions.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* fix format specifier

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-11 10:42:46 +09:00
Hiroshi Miura
7148599017 Default to use internal extraction (#80)
* Default to use py7zr
* Bamp up required py7zr >=0.5b5
* Drop --internal cli option

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-01-11 00:51:25 +09:00
Hiroshi Miura
97e9af09b3 Revert mirror selection to original
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-29 21:01:24 +09:00
Hiroshi Miura
fbf413be50 Improve mirror handling
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 21:56:41 +09:00
Hiroshi Miura
48660900e3 Use settings module for blacklisting
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 12:55:51 +09:00
Hiroshi Miura
17402cdde5 update qconfig.pri licheck (#58)
Fix #57

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-17 20:51:41 +09:00
Hiroshi Miura
a26f841882 Require py7zr only for py>3.5
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-16 20:11:33 +09:00
Hiroshi Miura
0363599359 Change install path (#48) (#49)
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-10 13:25:46 +09:00
Hiroshi Miura
e937dff283 Add CI test for tool installation (#47)
* Test: Update target Qt versions
* Add tools test
* tools: fix errors

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-08 09:26:24 +09:00
Hiroshi Miura
b6a294df0d PEP8
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-07 14:16:42 +09:00