deskflow/scripts/lib/env.py

172 lines
4.8 KiB
Python
Raw Normal View History

Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
import os, sys, subprocess, platform
from lib import env, cmd_utils
venv_path = "build/python"
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
version_file = "VERSION"
version_env_var = "SYNERGY_VERSION"
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
def check_module(module):
try:
__import__(module)
return True
except ImportError:
print(f"Python is missing {module} module", file=sys.stderr)
return False
def get_os():
"""Detects the operating system."""
if sys.platform == "win32":
return "windows"
elif sys.platform == "darwin":
return "mac"
elif sys.platform.startswith("linux"):
return "linux"
else:
raise RuntimeError(f"Unsupported platform: {sys.platform}")
def is_windows():
return get_os() == "windows"
def is_mac():
return get_os() == "mac"
def is_linux():
return get_os() == "linux"
def is_running_in_ci():
"""Returns True if running in a CI environment."""
return os.environ.get("CI")
def get_linux_distro():
"""Detects the Linux distro."""
os_file = "/etc/os-release"
if os.path.isfile(os_file):
with open(os_file) as f:
for line in f:
if line.startswith("ID="):
return line.strip().split("=")[1].strip('"')
return None
def get_env_var(name):
"""Returns an env var or raises an error if it is not set."""
value = os.getenv(name)
if not value:
raise ValueError(f"Environment variable not set: {name}")
return value
def get_python_executable(binary="python"):
if sys.platform == "win32":
return os.path.join(venv_path, "Scripts", binary)
else:
return os.path.join(venv_path, "bin", binary)
def in_venv():
"""Returns True if the script is running in a Python virtual environment."""
return sys.prefix != sys.base_prefix
def ensure_in_venv(script):
"""
Ensures the script is running in a Python virtual environment (venv).
If the script is not running in a venv, it will create one and re-run the script in the venv.
"""
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
assert_dependencies()
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
import venv
if not in_venv():
if not os.path.exists(venv_path):
print(f"Creating virtual environment at {venv_path}")
venv.create(venv_path, with_pip=True)
script_file = os.path.basename(script)
print(f"Using virtual environment for {script_file}")
sys.stdout.flush()
python_executable = get_python_executable()
result = subprocess.run([python_executable, script] + sys.argv[1:])
sys.exit(result.returncode)
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
# TODO: Use requirements.txt or pyproject.toml to specify dependencies
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
def ensure_module(module, package):
"""
Ensures that a Python module is available, and installs the package if it is not.
"""
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
assert_dependencies()
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
try:
__import__(module)
except ImportError:
print(f"Python missing {module}, installing {package}...", file=sys.stderr)
cmd_utils.run([sys.executable, "-m", "pip", "install", package], shell=False)
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
def assert_dependencies(raise_error=True):
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
"""
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
Returns True if pip and venv are available.
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
"""
has_pip = check_module("pip")
has_venv = check_module("venv")
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
if raise_error:
if not has_pip:
raise RuntimeError("Python is missing pip")
if not has_venv:
raise RuntimeError("Python is missing venv")
else:
return has_pip and has_venv
def ensure_dependencies():
"""
Ensures that pip and venv are available, and installs them if they are not.
This is normally only installs on Linux, as Windows and Mac usually come with pip and venv.
"""
if assert_dependencies(raise_error=False):
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
return
print("Installing Python dependencies...")
os = get_os()
if os != "linux":
# should not be a problem, since windows and mac come with pip and venv
raise RuntimeError(f"Unable to install Python dependencies on {os}")
has_sudo = cmd_utils.has_command("sudo")
sudo = "sudo" if has_sudo else ""
distro = get_linux_distro()
if distro == "ubuntu" or distro == "debian":
cmd_utils.run(f"{sudo} apt update".strip(), check=False)
cmd_utils.run(f"{sudo} apt install -y python3-pip python3-venv".strip())
elif distro == "fedora" or distro == "centos":
cmd_utils.run(f"{sudo} dnf check-update".strip(), check=False)
cmd_utils.run(f"{sudo} dnf install -y python3-pip python3-virtualenv".strip())
else:
# arch, opensuse, etc, patches welcome! :)
raise RuntimeError(f"Unable to install Python dependencies on {distro}")
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
def get_app_version():
"""
Returns the version either from the env var, or from the version file.
"""
if version_env_var in os.environ:
return os.environ[version_env_var]
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
Re-implement CI auto version increment for packaging (#7354) * Only install Python deps with deps script * Remove hard-coded Qt version in CI * Add example .env * Improve comments in .env file * Simplify config reader and Choco CI config * Actually return the config value * Move deps before Qt version env call * Remove `self.` * Move venv ensure to main * Fixed arg * Move config import to function * Move ensure_module to function * Simplify over-complicated `Version.cmake` * Move code to `github` module * Use `symless/next-version-action` * Make CMake version file even simpler * Set version from tag * Create release when master or release branch * Don't run CI on master push, only release * Fixed bac macro * Use new version values * Handle empty version env var * Also strip version file value * Remove quotes * Add @master to action * Read version from file * Simplify version file read * Fixed typo in env var * Remove unused var * Delete legacy build number action * Fixed env var * Version file read shouldn't be needed * Remove weird and unnecessary include path * Update ChangeLog * Remove unused config value * Better name for changelog check * Delete broken Flatpak CI * Run stale issue cron at midnight * Update ChangeLog * Add version input for manual run * Print next version * Fetch all tags * Use more idiomatic approach * Set to pre-release when master * Remove unnecessary `commitish` * Fixed wrong ID * More specific IDs * Reduce config needed for upload * Only /release can be non-pre-release * More discreet package path for gdrive * Try without setting path * Remove seemly unused step * Better name for Qt version step * Fixed bad var ref * Better name for package dir * Fixed bad input name * Add missing shell * Workflow to test upload action * Remove fetch tags * Use bash to cut version * Remove name and make conditional * Replace deprecated set-output * Fixed env var name * Missing dir sep char * Add comment to test workflow * Improve input descriptions * Replace deprecated set-output
2024-06-26 15:46:42 +00:00
with open(version_file, "r") as f:
return f.read().strip()