deskflow/scripts/lib/cmd_utils.py

128 lines
4.5 KiB
Python
Raw Normal View History

import subprocess
import sys
Re-implement packaging for GitHub workflows (Linux) (#7361) * Reorganize CMake Packaging module * Match if statements to function order * Cleanup root, res, and doc dirs * Move deps to requirements.txt file * Reorganize and format CMake files * Rename changelog lint * Add reccomended extension * Workflow to lint CMake files * Move CMake lint to script * Try lighter dep * Use venv * Add --format arg * Format all CMake files * Convert bash script to Python * Set CMake line ending format * Restore formatting * Add pyyaml dep * Remove unused arg * Rename config file * Remove comment * Repair copyrights (broken by defualt cmake-format) * Restore 3rd party copyright * Break up libs config into smaller macros * Better macro name * Load config after venv * Make intentional noop clearer * Only use upload step if required (make skip clearer) * Use CPack for deb and rpm packaging * Add upload step for Linux * Remove cpack dep, doesn't exist * Roll back presets version * Fixed distro like match * Update ChangeLog * Legacy checkout for some distros * All distros support v4 * Trying out newer Linux distros * Install Git on Docker images * Install without actions (not available before checkout) * Delete useless action * Install Python * Support for Arch and OpenSUSE * Add Arch and OpenSUSE to deps * Name steps * Full OpenSUSE names * Mark Git dir safe * Add pkgconf * Legacy CMake for Debian 11 * Add OpenSSL to OpenSUSE * Drop OpenSUSE Leap (no C++20 support) * Skip packaging for Arch and OpenSUSE (for now) * Shorten Arch/OpenSUSE names * Clearer step name * SImpler bootstrap * Shell not needed * Update apt * Don't check return code * Simplify python deps commands * Add STGZ/.sh package type * Prevent input prompt * Only config git safe dir when needed * Try cache v4 * Safe dir for Ubuntu * Safe dir for Arch * All Docker images seem to need safe dir config * Refactor env var getters * Make Ubuntu build extra packages * Condense bootstep to single step * Fixed var name * Fixed bootstrap logic * Simplify logic for upload condition (Windows and macOS) * Make package/upload condition easier to understand * Add Manjaro * Generic names for Linux .tar.gz and .sh packages * Add Manjaro deps * Swap macOS matrix entries * Add Red Hat UBI * Remove RHEL subscription manager * Throw on unsupported package distro * Conditionally install pip and venv * Remove extra pip arg * Add config for RHEL * Install EPEL for RHEL * Back-out RHEL as EPEL requires subscription * Restore Python deps logic * Fixed bug: Packacking run twice * Testing arm32v7 and arm64v8 * Revert "Testing arm32v7 and arm64v8" This reverts commit cb3caf188d2b79ed083a62fc091de295f9889f3d. * Re-add icon and shortcut file for Linux to package * Support OpenSUSE RPM build * Check return code * Add `rpm-build` for OpenSUSE * Reorg packages * Remove busybox-which * Add --non-interactive * Move --non-interactive to correct position * Experiment with makepkg * Check and print package commands * Make distro version optional * Use 8 cores to build * Default to distro name only * Fixed bad PKGBUILD filename * Use 4-part version for Arch * Remove comma from conflicts * Use .tar.gz from cwd * Generate checksum for Arch * Fixed file extension * Use shell to print output * Don't use shell * Gaurd against bad cmd_utils.run * Fixed bad import * Use list command * Fixed unable to run list commands * Use source file name * Simplify PKGBUILD to use make install * Change install prefix * Use DESTDIR * Copy .desktop and .png to build dir * Restore original `install(FILES...` * Improving comments * Fixed: makepkg runs from `src` by default * Move error after command print * Remove shell arg * Package as a user instead of root (makepkg can't run as root) * Fixed codesign runs in shell * Allow list commands in shell on windows * Don't use sudo on arch * Install sudo on Arch * Fixed typo * Fix ownership of build directory for package user * Improve example .env * Change to depend on libstdc++6 * Add TODO * Fixed Fedora version * Remove libstdc++ deps (names vary between distros) * Roll back to Fedora 40 and 39 * Improve comment * Remove unneccesary default
2024-07-02 10:23:56 +00:00
import lib.env as env
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
def has_command(command):
platform = sys.platform
if platform == "win32":
cmd = f"where {command}"
else:
cmd = f"which {command}"
try:
subprocess.check_output(cmd, shell=True)
return True
except subprocess.CalledProcessError:
return False
def strip_continuation_sequences(command):
"""
Remove the continuation sequences (\\) from a command.
To spread strings over multiple lines in YAML files, like in bash, a backslash is used at
the end of each line as continuation character.
When a YAML file is parsed, this becomes "\\ " (without a new line char), so this character
sequence must be removed before running the command.
This doesn't seem to be an issue on Windows, since the \\ path separator is rarely followed
by a space.
"""
cmd_continuation = "\\ "
if isinstance(command, list):
return [c.replace(cmd_continuation, "") for c in command]
else:
return command.replace(cmd_continuation, "")
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
def run(
command,
check=True, # true by default to fail fast
shell=False, # false by default for security
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
get_output=False,
print_cmd=False, # false by default for security
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
):
"""
Convenience wrapper around `subprocess.run` to:
- print the command before running it (if `print_cmd` is True)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
This differs to `subprocess.run` in that by default it:
- checks the return code by default
- prints list commands as a readable string on failure
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
This is the same as `subprocess.run` in that it:
- does not use shell by default for security (shell is less secure)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
Args:
command (str or list): The command to run.
check (bool): Raise an exception if the command fails.
shell (bool): Run the command in a shell (false by default for security)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
get_output (bool): Return the output of the command.
print_cmd (bool): Print the command before running it (false by default for security)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
"""
Re-implement packaging for GitHub workflows (Linux) (#7361) * Reorganize CMake Packaging module * Match if statements to function order * Cleanup root, res, and doc dirs * Move deps to requirements.txt file * Reorganize and format CMake files * Rename changelog lint * Add reccomended extension * Workflow to lint CMake files * Move CMake lint to script * Try lighter dep * Use venv * Add --format arg * Format all CMake files * Convert bash script to Python * Set CMake line ending format * Restore formatting * Add pyyaml dep * Remove unused arg * Rename config file * Remove comment * Repair copyrights (broken by defualt cmake-format) * Restore 3rd party copyright * Break up libs config into smaller macros * Better macro name * Load config after venv * Make intentional noop clearer * Only use upload step if required (make skip clearer) * Use CPack for deb and rpm packaging * Add upload step for Linux * Remove cpack dep, doesn't exist * Roll back presets version * Fixed distro like match * Update ChangeLog * Legacy checkout for some distros * All distros support v4 * Trying out newer Linux distros * Install Git on Docker images * Install without actions (not available before checkout) * Delete useless action * Install Python * Support for Arch and OpenSUSE * Add Arch and OpenSUSE to deps * Name steps * Full OpenSUSE names * Mark Git dir safe * Add pkgconf * Legacy CMake for Debian 11 * Add OpenSSL to OpenSUSE * Drop OpenSUSE Leap (no C++20 support) * Skip packaging for Arch and OpenSUSE (for now) * Shorten Arch/OpenSUSE names * Clearer step name * SImpler bootstrap * Shell not needed * Update apt * Don't check return code * Simplify python deps commands * Add STGZ/.sh package type * Prevent input prompt * Only config git safe dir when needed * Try cache v4 * Safe dir for Ubuntu * Safe dir for Arch * All Docker images seem to need safe dir config * Refactor env var getters * Make Ubuntu build extra packages * Condense bootstep to single step * Fixed var name * Fixed bootstrap logic * Simplify logic for upload condition (Windows and macOS) * Make package/upload condition easier to understand * Add Manjaro * Generic names for Linux .tar.gz and .sh packages * Add Manjaro deps * Swap macOS matrix entries * Add Red Hat UBI * Remove RHEL subscription manager * Throw on unsupported package distro * Conditionally install pip and venv * Remove extra pip arg * Add config for RHEL * Install EPEL for RHEL * Back-out RHEL as EPEL requires subscription * Restore Python deps logic * Fixed bug: Packacking run twice * Testing arm32v7 and arm64v8 * Revert "Testing arm32v7 and arm64v8" This reverts commit cb3caf188d2b79ed083a62fc091de295f9889f3d. * Re-add icon and shortcut file for Linux to package * Support OpenSUSE RPM build * Check return code * Add `rpm-build` for OpenSUSE * Reorg packages * Remove busybox-which * Add --non-interactive * Move --non-interactive to correct position * Experiment with makepkg * Check and print package commands * Make distro version optional * Use 8 cores to build * Default to distro name only * Fixed bad PKGBUILD filename * Use 4-part version for Arch * Remove comma from conflicts * Use .tar.gz from cwd * Generate checksum for Arch * Fixed file extension * Use shell to print output * Don't use shell * Gaurd against bad cmd_utils.run * Fixed bad import * Use list command * Fixed unable to run list commands * Use source file name * Simplify PKGBUILD to use make install * Change install prefix * Use DESTDIR * Copy .desktop and .png to build dir * Restore original `install(FILES...` * Improving comments * Fixed: makepkg runs from `src` by default * Move error after command print * Remove shell arg * Package as a user instead of root (makepkg can't run as root) * Fixed codesign runs in shell * Allow list commands in shell on windows * Don't use sudo on arch * Install sudo on Arch * Fixed typo * Fix ownership of build directory for package user * Improve example .env * Change to depend on libstdc++6 * Add TODO * Fixed Fedora version * Remove libstdc++ deps (names vary between distros) * Roll back to Fedora 40 and 39 * Improve comment * Remove unneccesary default
2024-07-02 10:23:56 +00:00
is_list_cmd = isinstance(command, list)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
# create string version of list command, only for debugging purposes
command_str = command
Re-implement packaging for GitHub workflows (Linux) (#7361) * Reorganize CMake Packaging module * Match if statements to function order * Cleanup root, res, and doc dirs * Move deps to requirements.txt file * Reorganize and format CMake files * Rename changelog lint * Add reccomended extension * Workflow to lint CMake files * Move CMake lint to script * Try lighter dep * Use venv * Add --format arg * Format all CMake files * Convert bash script to Python * Set CMake line ending format * Restore formatting * Add pyyaml dep * Remove unused arg * Rename config file * Remove comment * Repair copyrights (broken by defualt cmake-format) * Restore 3rd party copyright * Break up libs config into smaller macros * Better macro name * Load config after venv * Make intentional noop clearer * Only use upload step if required (make skip clearer) * Use CPack for deb and rpm packaging * Add upload step for Linux * Remove cpack dep, doesn't exist * Roll back presets version * Fixed distro like match * Update ChangeLog * Legacy checkout for some distros * All distros support v4 * Trying out newer Linux distros * Install Git on Docker images * Install without actions (not available before checkout) * Delete useless action * Install Python * Support for Arch and OpenSUSE * Add Arch and OpenSUSE to deps * Name steps * Full OpenSUSE names * Mark Git dir safe * Add pkgconf * Legacy CMake for Debian 11 * Add OpenSSL to OpenSUSE * Drop OpenSUSE Leap (no C++20 support) * Skip packaging for Arch and OpenSUSE (for now) * Shorten Arch/OpenSUSE names * Clearer step name * SImpler bootstrap * Shell not needed * Update apt * Don't check return code * Simplify python deps commands * Add STGZ/.sh package type * Prevent input prompt * Only config git safe dir when needed * Try cache v4 * Safe dir for Ubuntu * Safe dir for Arch * All Docker images seem to need safe dir config * Refactor env var getters * Make Ubuntu build extra packages * Condense bootstep to single step * Fixed var name * Fixed bootstrap logic * Simplify logic for upload condition (Windows and macOS) * Make package/upload condition easier to understand * Add Manjaro * Generic names for Linux .tar.gz and .sh packages * Add Manjaro deps * Swap macOS matrix entries * Add Red Hat UBI * Remove RHEL subscription manager * Throw on unsupported package distro * Conditionally install pip and venv * Remove extra pip arg * Add config for RHEL * Install EPEL for RHEL * Back-out RHEL as EPEL requires subscription * Restore Python deps logic * Fixed bug: Packacking run twice * Testing arm32v7 and arm64v8 * Revert "Testing arm32v7 and arm64v8" This reverts commit cb3caf188d2b79ed083a62fc091de295f9889f3d. * Re-add icon and shortcut file for Linux to package * Support OpenSUSE RPM build * Check return code * Add `rpm-build` for OpenSUSE * Reorg packages * Remove busybox-which * Add --non-interactive * Move --non-interactive to correct position * Experiment with makepkg * Check and print package commands * Make distro version optional * Use 8 cores to build * Default to distro name only * Fixed bad PKGBUILD filename * Use 4-part version for Arch * Remove comma from conflicts * Use .tar.gz from cwd * Generate checksum for Arch * Fixed file extension * Use shell to print output * Don't use shell * Gaurd against bad cmd_utils.run * Fixed bad import * Use list command * Fixed unable to run list commands * Use source file name * Simplify PKGBUILD to use make install * Change install prefix * Use DESTDIR * Copy .desktop and .png to build dir * Restore original `install(FILES...` * Improving comments * Fixed: makepkg runs from `src` by default * Move error after command print * Remove shell arg * Package as a user instead of root (makepkg can't run as root) * Fixed codesign runs in shell * Allow list commands in shell on windows * Don't use sudo on arch * Install sudo on Arch * Fixed typo * Fix ownership of build directory for package user * Improve example .env * Change to depend on libstdc++6 * Add TODO * Fixed Fedora version * Remove libstdc++ deps (names vary between distros) * Roll back to Fedora 40 and 39 * Improve comment * Remove unneccesary default
2024-07-02 10:23:56 +00:00
if is_list_cmd:
command_str = " ".join(command)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
if print_cmd:
print(f"Running: {command_str}")
else:
print("Running command...")
command_str = "***"
Re-implement packaging for GitHub workflows (Linux) (#7361) * Reorganize CMake Packaging module * Match if statements to function order * Cleanup root, res, and doc dirs * Move deps to requirements.txt file * Reorganize and format CMake files * Rename changelog lint * Add reccomended extension * Workflow to lint CMake files * Move CMake lint to script * Try lighter dep * Use venv * Add --format arg * Format all CMake files * Convert bash script to Python * Set CMake line ending format * Restore formatting * Add pyyaml dep * Remove unused arg * Rename config file * Remove comment * Repair copyrights (broken by defualt cmake-format) * Restore 3rd party copyright * Break up libs config into smaller macros * Better macro name * Load config after venv * Make intentional noop clearer * Only use upload step if required (make skip clearer) * Use CPack for deb and rpm packaging * Add upload step for Linux * Remove cpack dep, doesn't exist * Roll back presets version * Fixed distro like match * Update ChangeLog * Legacy checkout for some distros * All distros support v4 * Trying out newer Linux distros * Install Git on Docker images * Install without actions (not available before checkout) * Delete useless action * Install Python * Support for Arch and OpenSUSE * Add Arch and OpenSUSE to deps * Name steps * Full OpenSUSE names * Mark Git dir safe * Add pkgconf * Legacy CMake for Debian 11 * Add OpenSSL to OpenSUSE * Drop OpenSUSE Leap (no C++20 support) * Skip packaging for Arch and OpenSUSE (for now) * Shorten Arch/OpenSUSE names * Clearer step name * SImpler bootstrap * Shell not needed * Update apt * Don't check return code * Simplify python deps commands * Add STGZ/.sh package type * Prevent input prompt * Only config git safe dir when needed * Try cache v4 * Safe dir for Ubuntu * Safe dir for Arch * All Docker images seem to need safe dir config * Refactor env var getters * Make Ubuntu build extra packages * Condense bootstep to single step * Fixed var name * Fixed bootstrap logic * Simplify logic for upload condition (Windows and macOS) * Make package/upload condition easier to understand * Add Manjaro * Generic names for Linux .tar.gz and .sh packages * Add Manjaro deps * Swap macOS matrix entries * Add Red Hat UBI * Remove RHEL subscription manager * Throw on unsupported package distro * Conditionally install pip and venv * Remove extra pip arg * Add config for RHEL * Install EPEL for RHEL * Back-out RHEL as EPEL requires subscription * Restore Python deps logic * Fixed bug: Packacking run twice * Testing arm32v7 and arm64v8 * Revert "Testing arm32v7 and arm64v8" This reverts commit cb3caf188d2b79ed083a62fc091de295f9889f3d. * Re-add icon and shortcut file for Linux to package * Support OpenSUSE RPM build * Check return code * Add `rpm-build` for OpenSUSE * Reorg packages * Remove busybox-which * Add --non-interactive * Move --non-interactive to correct position * Experiment with makepkg * Check and print package commands * Make distro version optional * Use 8 cores to build * Default to distro name only * Fixed bad PKGBUILD filename * Use 4-part version for Arch * Remove comma from conflicts * Use .tar.gz from cwd * Generate checksum for Arch * Fixed file extension * Use shell to print output * Don't use shell * Gaurd against bad cmd_utils.run * Fixed bad import * Use list command * Fixed unable to run list commands * Use source file name * Simplify PKGBUILD to use make install * Change install prefix * Use DESTDIR * Copy .desktop and .png to build dir * Restore original `install(FILES...` * Improving comments * Fixed: makepkg runs from `src` by default * Move error after command print * Remove shell arg * Package as a user instead of root (makepkg can't run as root) * Fixed codesign runs in shell * Allow list commands in shell on windows * Don't use sudo on arch * Install sudo on Arch * Fixed typo * Fix ownership of build directory for package user * Improve example .env * Change to depend on libstdc++6 * Add TODO * Fixed Fedora version * Remove libstdc++ deps (names vary between distros) * Roll back to Fedora 40 and 39 * Improve comment * Remove unneccesary default
2024-07-02 10:23:56 +00:00
# TODO: You can definitely use a list command with shell=True on Windows,
# but can you use a string command with shell=False on Windows?
#
# The `subprocess.run` function has a little gotcha:
# - a string command must be used when `shell=True`
# - a list command must be used when shell isn't or `shell=False`
# however, it allows you to pass a string command when shell isn't used or `shell=False`
# then fails with a vague error message. same problem with list commands and `shell=True`
if not env.is_windows() and is_list_cmd and shell:
raise ValueError("List commands cannot be used when shell=True on Unix systems")
elif not is_list_cmd and not shell:
raise ValueError("String commands cannot be used when shell=False or not set")
# Flush the output to ensure the command is printed before the output of the command,
# which seems to happen in the GitHub runner logs.
sys.stdout.flush()
sys.stderr.flush()
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
try:
if get_output:
result = subprocess.run(
command,
shell=shell,
check=check,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
else:
result = subprocess.run(command, check=check, shell=shell)
Re-implement packaging for GitHub workflows (Linux) (#7361) * Reorganize CMake Packaging module * Match if statements to function order * Cleanup root, res, and doc dirs * Move deps to requirements.txt file * Reorganize and format CMake files * Rename changelog lint * Add reccomended extension * Workflow to lint CMake files * Move CMake lint to script * Try lighter dep * Use venv * Add --format arg * Format all CMake files * Convert bash script to Python * Set CMake line ending format * Restore formatting * Add pyyaml dep * Remove unused arg * Rename config file * Remove comment * Repair copyrights (broken by defualt cmake-format) * Restore 3rd party copyright * Break up libs config into smaller macros * Better macro name * Load config after venv * Make intentional noop clearer * Only use upload step if required (make skip clearer) * Use CPack for deb and rpm packaging * Add upload step for Linux * Remove cpack dep, doesn't exist * Roll back presets version * Fixed distro like match * Update ChangeLog * Legacy checkout for some distros * All distros support v4 * Trying out newer Linux distros * Install Git on Docker images * Install without actions (not available before checkout) * Delete useless action * Install Python * Support for Arch and OpenSUSE * Add Arch and OpenSUSE to deps * Name steps * Full OpenSUSE names * Mark Git dir safe * Add pkgconf * Legacy CMake for Debian 11 * Add OpenSSL to OpenSUSE * Drop OpenSUSE Leap (no C++20 support) * Skip packaging for Arch and OpenSUSE (for now) * Shorten Arch/OpenSUSE names * Clearer step name * SImpler bootstrap * Shell not needed * Update apt * Don't check return code * Simplify python deps commands * Add STGZ/.sh package type * Prevent input prompt * Only config git safe dir when needed * Try cache v4 * Safe dir for Ubuntu * Safe dir for Arch * All Docker images seem to need safe dir config * Refactor env var getters * Make Ubuntu build extra packages * Condense bootstep to single step * Fixed var name * Fixed bootstrap logic * Simplify logic for upload condition (Windows and macOS) * Make package/upload condition easier to understand * Add Manjaro * Generic names for Linux .tar.gz and .sh packages * Add Manjaro deps * Swap macOS matrix entries * Add Red Hat UBI * Remove RHEL subscription manager * Throw on unsupported package distro * Conditionally install pip and venv * Remove extra pip arg * Add config for RHEL * Install EPEL for RHEL * Back-out RHEL as EPEL requires subscription * Restore Python deps logic * Fixed bug: Packacking run twice * Testing arm32v7 and arm64v8 * Revert "Testing arm32v7 and arm64v8" This reverts commit cb3caf188d2b79ed083a62fc091de295f9889f3d. * Re-add icon and shortcut file for Linux to package * Support OpenSUSE RPM build * Check return code * Add `rpm-build` for OpenSUSE * Reorg packages * Remove busybox-which * Add --non-interactive * Move --non-interactive to correct position * Experiment with makepkg * Check and print package commands * Make distro version optional * Use 8 cores to build * Default to distro name only * Fixed bad PKGBUILD filename * Use 4-part version for Arch * Remove comma from conflicts * Use .tar.gz from cwd * Generate checksum for Arch * Fixed file extension * Use shell to print output * Don't use shell * Gaurd against bad cmd_utils.run * Fixed bad import * Use list command * Fixed unable to run list commands * Use source file name * Simplify PKGBUILD to use make install * Change install prefix * Use DESTDIR * Copy .desktop and .png to build dir * Restore original `install(FILES...` * Improving comments * Fixed: makepkg runs from `src` by default * Move error after command print * Remove shell arg * Package as a user instead of root (makepkg can't run as root) * Fixed codesign runs in shell * Allow list commands in shell on windows * Don't use sudo on arch * Install sudo on Arch * Fixed typo * Fix ownership of build directory for package user * Improve example .env * Change to depend on libstdc++6 * Add TODO * Fixed Fedora version * Remove libstdc++ deps (names vary between distros) * Roll back to Fedora 40 and 39 * Improve comment * Remove unneccesary default
2024-07-02 10:23:56 +00:00
except subprocess.CalledProcessError as e:
# Take control of how failed commands are printed:
# - if `print_cmd` is false, it will print `***` instead of the command
# - if the command was a list, the command is printed as a readable string
raise RuntimeError(
f"Command exited with code {e.returncode}: {command_str}"
) from None
except Exception:
# Take control of how failed commands are printed:
# - if `print_cmd` is false, it will print `***` instead of the command
# - if the command was a list, the command is printed as a readable string
raise RuntimeError(f"Command failed: {command_str}") from None
if result.returncode != 0:
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
print(
f"Command exited with code {result.returncode}: {command_str}",
file=sys.stderr,
)
Re-implement packaging for GitHub workflows (macOS) (#7353) * Restore Azure macOS dist scripts * Move steps to workflow for testing * Always upload to GitHub * Add codesign ID * Echo codesign ID * Add cert import code * Stub file for Mac * Self-install pyyaml and choco * Auto add env var on Windows * Auto add CMAKE_PREFIX_PATH to .zshrc * Shorter var names * Append env var instead of replace * Only set env var if not CI * Improve function names and print output * Simplify Linux package command * Support continuation sequence * Add note about Windows * Remove dead doc file * Tidy up version file and move to .env format * Use Python venv for deps * Only use venv on Mac * Rename package script for all OS * Add package and dist steps, and use common upload * Remove version source * Fixed vars not available * Fixed python paths * Use RuntimeError which is sufficient * Remove dead code * Add extras command for Linux * Always install deps on Linux * Move Python deps to CI * More env bootstrapping, ugh * Forgot to return! * Simplify code * Use shell * Simplify command * Skip sudo if no sudo * Update package managers * Fixed Fedora package name * Tidy up commands * Use newer upload artifact * Strip don't trim! * Check for version file and reduce log verbosity * Remove CentOS 7.6 * Print more info about return code and log more to stderr * Install certificate on macOS * Better errors for no env var * Implement Mac signing and notary * Move dmgbuild load * Simplify notary * Rename dist files to same as dest * Fixed paths for dist * Move checked-in dist files to res (dist is meant to be a temp dir) * Fixed Mac path in CMake * Fixed dmg path * Format Python * Ignore import warnings and move function * Fixed cmake paths * Add missing env var secrets * Remove extensions from GH upload * Make deps.yml general purpose config * Add cspell config * Pass codesign ID * Use new general config file * Sign bundle on Mac * Move imports to functions * Escape chars in docs * Fixed config key accessor * Change module import order * Move file to tmp dir in workflow dir * Persist temp dir * Add tmp dir to ignore * Flush stdio before running process * Trying quotes around env values * Add codesigning certificate validation for Mac signing * Revert "Trying quotes around env values" This reverts commit 0dd741e8cd6fde21e69d4fb871e835a5f4fa1a23. * Extract codesign verify * Fixed version number * Ignore .cache dir * Fix macro name * Package name with version number and arch * Improve package function readability * Change order of vars * Testing upload to GDrive * Add missing return code * Use positional args and declare error * Use machine instead of arch and remove build from filename * Remove redundant build jobs * Replace massively over-complicated `build_version.py` script * Move version info to env module * Use version info script * Fixed: too many values to unpack * Chmod version script * Use shebang * Don't check return code on Linux * Fixed function name * Convert to GitHub specific script * Env vars must be after configure * Fixed Windows env var command * Remove && from deps command so it's not conditional * Fixed position of set env * Change order of env script * Only upload when not draft * Test * Tweak config * Fixed if condition * Don't package in draft (Windows and Linux)
2024-06-24 09:36:30 +00:00
return result