Commit Graph

66 Commits

Author SHA1 Message Date
Hiroshi Miura
fcb5b45631 CLI: Add --archives option
when specify '--archive qtbase qttools' then
aqt installs these subset of installation.

There is no gurantee it works, and it is an advanced
option.

Implemented #126

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-19 12:14:25 +09:00
Hiroshi Miura
bdb1156351 py7zr depndency bamp up to v0.7.0 (#125)
* Configure all dependency on setup.cfg
* depend on py7zr 0.7

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-09 23:26:33 +09:00
Hiroshi Miura
188b32de8f Fix to use concurrency settings
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-07 13:46:13 +09:00
Hiroshi Miura
caf804f3ae Limit retries with back-off factor
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-06 08:22:33 +09:00
Hiroshi Miura
3b06814601 Refactoring altlink helper (#109)
* Refactoring altlink helper

Also fix a bug not accept a header context-type when
got charset parameter such as
application/meta4+xml; charset=utf-8

* Add test and accept text/plain mime type

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-05 23:00:31 +09:00
Hiroshi Miura
beb755219a Check redirect location before get metalink data
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-04 16:30:16 +09:00
Hiroshi Miura
79b5a297b5 Check version number and improve error messages (#102)
* Check Qt versions and report 404 error

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-01 11:35:39 +09:00
Hiroshi Miura
36238fc007 isort
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-28 08:56:03 +09:00
Kyle Altendorf
0d929826d4 no more os.chdir() 2020-02-26 00:13:39 -05:00
Hiroshi Miura
8b983d0a65 Work around a bug of py7zr may change current working directory
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:32:58 +09:00
Hiroshi Miura
705920b1cc Revert "Use written path as pathlib.Path"
This reverts commit 134ec36dc6.
2020-02-26 10:30:37 +09:00
Hiroshi Miura
134ec36dc6 Use written path as pathlib.Path
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:13:42 +09:00
Hiroshi Miura
ebead77b65 add fd.flush() to writing files
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 10:05:32 +09:00
Hiroshi Miura
42078a2834 Pass opened file object to extractor
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 09:47:42 +09:00
Hiroshi Miura
3d945ba9f6 Refactoring threading of installation
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-26 08:31:15 +09:00
Hiroshi Miura
58d8340959 Use concurrent.futures for threading as same as v0.7.*
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-24 10:05:53 +09:00
Hiroshi Miura
81e55518d5 Fix installation logics
- Fix condition to start extraction which leads FileNotFound error.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 15:43:23 +09:00
Hiroshi Miura
25623901e5 Fix extraction check
- An order of list of completed task become as same
  both download and extract.
  This will fix a FileNotFound error.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 14:19:41 +09:00
Hiroshi Miura
6dd63186d8 Get exception on each threads
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-23 11:57:28 +09:00
Hiroshi Miura
527ce3fd4f Test with py7zr 0.6.*
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-22 09:59:46 +09:00
Hiroshi Miura
4b18f6520d Limit extraction thread number to 4
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 16:08:09 +09:00
Hiroshi Miura
9f15361ace Use only threading for processing
- Now limiting download on fixed concurrency with threading.
- Extraction is unlimited threading, but extraction external process or
  library may run on multiprocessing.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 16:03:12 +09:00
Hiroshi Miura
67e6e1c0cb use Popen for lauching external 7zip command
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-21 08:46:50 +09:00
Hiroshi Miura
bbb39c0847 Use concurrent.futures for concurrency with thread and process (#87)
Use concurrent.futures for multiprocessing and other improvements
- combination with thread pool for downloading and  process pool for extraction.
- Iterate all download and start extraction.
- Start extraction when download completed
- Add download completion log
- log a time  when extraction done
- Show elasped time for installation
- Meature elasped time
  * use perf_counter() for a total elapsed time.
  * use process_time() for individual extractions.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* fix format specifier

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-02-11 10:42:46 +09:00
Hiroshi Miura
7148599017 Default to use internal extraction (#80)
* Default to use py7zr
* Bamp up required py7zr >=0.5b5
* Drop --internal cli option

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-01-11 00:51:25 +09:00
Hiroshi Miura
97e9af09b3 Revert mirror selection to original
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-29 21:01:24 +09:00
Hiroshi Miura
fbf413be50 Improve mirror handling
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 21:56:41 +09:00
Hiroshi Miura
48660900e3 Use settings module for blacklisting
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 12:55:51 +09:00
Hiroshi Miura
17402cdde5 update qconfig.pri licheck (#58)
Fix #57

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-17 20:51:41 +09:00
Hiroshi Miura
a26f841882 Require py7zr only for py>3.5
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-16 20:11:33 +09:00
Hiroshi Miura
0363599359 Change install path (#48) (#49)
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-10 13:25:46 +09:00
Hiroshi Miura
e937dff283 Add CI test for tool installation (#47)
* Test: Update target Qt versions
* Add tools test
* tools: fix errors

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-08 09:26:24 +09:00
Hiroshi Miura
b6a294df0d PEP8
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-07 14:16:42 +09:00
Hiroshi Miura
346f2ce436 Experimental implementation for tools installer
- command line
  `python -m aqt tools tools_ifw windows desktop qt.tools.ifw.31`

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-07 14:11:24 +09:00
Hiroshi Miura
380f4a1948 Fix uninitialized variable use
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-10-04 17:08:56 +09:00
Hiroshi Miura
e8de2e7869 Add MANIFEST.in for distributing logging.yml
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-09-28 16:23:47 +09:00
Hiroshi Miura
886376d81b Add logging argument options
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-09-28 11:41:53 +09:00
Hiroshi Miura
7b1430fb39 Allow redirection on connection to mirror
fixed #41
2019-09-25 09:19:25 +09:00
Hiroshi Miura
4fa64b9f5d Use py7zr for extracting package 7z archive
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-04 09:52:20 +09:00
Hiroshi Miura
a7c56c87b7 Support installarion of mingw runtime
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-02 13:47:48 +09:00
Hiroshi Miura
6717aef6fa PEP8
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-02 10:17:06 +09:00
Hiroshi Miura
375fc81b7b Revert "Add more 7zip command option"
This reverts commit 9434873194.
2019-06-02 00:24:14 +09:00
Hiroshi Miura
0f54aa24b8 fix missing import
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 23:57:23 +09:00
Hiroshi Miura
9434873194 Add more 7zip command option
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 23:53:05 +09:00
Hiroshi Miura
44ec371c96 Always check miror site list for target
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 23:52:13 +09:00
Hiroshi Miura
a56933637f Refactoring metalink class
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 17:40:34 +09:00
Hiroshi Miura
e88bbb156b Fix redirection error
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 14:36:27 +09:00
Hiroshi Miura
74a3459773 Improve error message
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-06-01 14:31:39 +09:00
Hiroshi Miura
b34a8099e6 Add mirror url option
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-05-29 22:20:01 +09:00
Hiroshi Miura
39297ee72b Fix file not found error on win32_mingw73
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-05-29 07:43:12 +09:00