Commit Graph

32 Commits

Author SHA1 Message Date
David Dalcino
e835db919a Align logging levels in getUrl to downloadBinaryFile
PR #273 introduced some changes to logging levels in
`downloadBinaryFile`. This change alters `getUrl` to reflect those
changes.
2021-07-01 14:43:49 -07:00
David Dalcino
914cc92546 Remove invalid param from call to altlink 2021-07-01 14:39:06 -07:00
David Dalcino
39c6fd7922 Fix error message for non-200 status code
The original error message, `Download error when access <filename>`,
is confusing when a user is trying to use `aqt list`. This message
could make the user think that `aqt list` is attempting to download
archives, when in reality the command is only reading files on the
server.

Initially, I wanted to make this error message modifiable by the caller.
On further investigation, I realized that this function is always used
to retrieve 'Updates.xml' files or hashes, and that the error message
"Failed to retrieve file" would be appropriate for all usages.

Admittedly, the distinction between "Download error" and "Failed to
retrieve file" is subtle. However, the difference was big enough to
convince me as a developer that I should not use this function to
retrieve "Updates.xml" files, when in fact it was the right function to
use.
2021-07-01 13:57:30 -07:00
David Dalcino
5d69e87676 Fix multiple redirects in getUrl
This change handles multiple redirects. Without this change, if
`helper.altlink()` returns a url that will redirect, we cannot follow
that redirect. This change allows up to 10 such redirects to occur.

Please note that the upper bound for number of redirects was chosen
arbitrarily, with no regard for what might realistically occur. I felt
that it was important not to allow recursive or infinite redirects, and
that at least two redirects should be allowed. Perhaps the upper bound
should be modifiable in settings.ini.
2021-07-01 13:40:23 -07:00
David Dalcino
be48755bc5 Remove odd unicode characters from docstring
These characters caused Black to crash on my PC. I believe that these characters were committed by mistake; if I am wrong, please close this PR.
2021-06-29 11:29:10 -07:00
Hiroshi Miura
f35edf398a Fix logging in installer sub-process when multiprocessing context with spawn method (#273)
* Simplify logging

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Update logging

- keep logging file path in Settings
- introduce installer logger

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* PEP8/Black

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: add queue handler

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: introduce logutils module

- Add QueueListenerHandler
- Add queue listener handler config
- qualname to aqt.installer

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: introduce aqt.updater logger

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: introduce aqt.main and aqt.archives logger

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* PEP8/Black

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: simplify config file

- use propergate for last output handler
- change formatter
  * console: brief
  * file: verbose
- Use multiprocessing.Queue() instead of queue.Queue()
- Remember logging config file customization in Settings
- QueueListener use handler as of getLogger("aqt").handlers

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* change log level of redirection

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Logging: Single global LoggingQueueListener

- Global instance of LoggingQueueListener
- Drop queue_listener from logging.ini
- Generate QueueHandler from LoggingQueueLisnter.get_queue_handler
- Explicitly close QueueListenerHandler and Queue object.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: use verbose console logging for ci

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CLI: drop options to specify logger and logger configuration file

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Make Instance of LoggingQueueListener in Cli class

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Handle QueueListener in Cli.call_installer

- Use mp.Manager().Queue()
- start(), stop() in caller
- Gracefully close QueueHandler in installer

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* PEP8

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Drop berkeley.edu from fallbacks of mirror site

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Fix log format of time digits

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Update settings load

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: catch custom settings from change in master branch

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Show log when loading custom settings

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Improve subprocess logging that use root logger

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* fix logging.ini settings

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Fix against tests

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Introduce MyQueueListener

- Inherit System QueueListner to override logger name

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* PEP8/Black

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-06-21 15:25:00 +09:00
Hiroshi Miura
99b9c216a5 Inroduce --kde option for src subcommand
It apply KDE patch collection to Qt 5.15.2 source
`aqt src 5.15.2 linux desktop --kde`

Patch collection is based on Qt 5.15.2, so only the version can be patched.

Ref #231

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-06-19 15:02:08 +09:00
Hiroshi Miura
fac014216f Use global instance to hold Settings instead of Borg (#272)
* Use global variable to hold Settings

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Settings global instance to hold configurations

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Revert to Settings class to helper module

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Revert Settings class variables

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* PEP8

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* CI: change actions to use macos-11

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Revert "CI: change actions to use macos-11"

CI: macos-11 is just for private preview

This reverts commit f1d576f4bf.
2021-06-19 10:43:19 +09:00
Hiroshi Miura
18fd3da009 test for download utilities
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-25 13:05:02 +09:00
Hiroshi Miura
741eb4989e PEP8/Black
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-24 08:27:07 +09:00
Hiroshi Miura
06e2292153 helper: guarantee blacklist is not None
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 17:47:37 +09:00
Hiroshi Miura
df7222dd34 Better redirection handling
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 15:39:20 +09:00
Hiroshi Miura
fb088cac05 settings.ini: fix -c option handling
when argparse.FileType("r") used, variable passed is a open-ed file.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 15:16:38 +09:00
Hiroshi Miura
089cc3427c settings.ini: use multiline configuration for URL list
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 14:05:17 +09:00
Hiroshi Miura
4c0638250c Move constants into settings.ini configuration
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 13:00:31 +09:00
Hiroshi Miura
0cfeda838a PEP8/Black
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 12:12:52 +09:00
Hiroshi Miura
ca72e070c7 test: Early load custom configuration
hack for borg class

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 12:12:04 +09:00
Hiroshi Miura
dd74254ea0 Check config file existence and handle parsing error
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-23 11:30:30 +09:00
Hiroshi Miura
6b12ce9e9d Merge pull request #244 from miurahr/patch-add-blacklist
Refactoring network code
2021-05-22 21:25:52 +09:00
Hiroshi Miura
43cb99bc9c Refactoring network code
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-22 17:26:16 +09:00
Hiroshi Miura
53c71ff664 refactor Cli class (#243)
* refactor Cli class

- check version combination
- remove dead code

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* fix typo in json

Signed-off-by: Hiroshi Miura <miurahr@linux.com>

* Update version combinations

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-05-22 14:25:17 +09:00
Hiroshi Miura
d841741641 Reformat by black
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-04-02 12:20:10 +09:00
Hiroshi Miura
c2750e34f1 Update documents and copyrights
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-30 00:28:44 +09:00
Hiroshi Miura
655e6e2b66 Improve patching
- Patch pkgconfig configurations
- Patch libQt5Core and libQt6Core for linux
- Catch exception on qmake -query execution

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2021-01-27 14:46:06 +09:00
Hiroshi Miura
b05832db04 test: update tests
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 22:49:05 +09:00
Hiroshi Miura
d8b0ae647c Refactoring
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 22:35:37 +09:00
Hiroshi Miura
8bb70f6f37 Refactoring modules
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-11-21 21:17:24 +09:00
Hiroshi Miura
72c4589e7d Patch qmake as finalize process (#130)
* Patch qmake hard coded path with installed prefix(#100)
* AP: Update tests
 - Test android target with 5.14.x
 - Test ios target on mac
* Does not patch mac framework when android, ios and wasm
* Update changelog
* GHA: Update to run qmake to check patch
  - Check qmake works well if patched.

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-05-21 21:56:04 +09:00
Hiroshi Miura
3b06814601 Refactoring altlink helper (#109)
* Refactoring altlink helper

Also fix a bug not accept a header context-type when
got charset parameter such as
application/meta4+xml; charset=utf-8

* Add test and accept text/plain mime type

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-05 23:00:31 +09:00
Hiroshi Miura
beb755219a Check redirect location before get metalink data
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2020-03-04 16:30:16 +09:00
Hiroshi Miura
fbf413be50 Improve mirror handling
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 21:56:41 +09:00
Hiroshi Miura
48660900e3 Use settings module for blacklisting
Signed-off-by: Hiroshi Miura <miurahr@linux.com>
2019-11-24 12:55:51 +09:00