Commits on Source (25)
-
Matthew Clarkson authored
Missed the `inputs` key, oops.
-
Matthew Clarkson authored
-
Matthew Clarkson authored
-
Matthew Clarkson authored
-
Matthew Clarkson authored
Bazel does not invalidate downstream repository rules unless explicitly watched.
-
Matthew Clarkson authored
-
Matthew Clarkson authored
Helps make `MODULE.bazel` more consistent and explicit
-
Matthew Clarkson authored
Keeps the consistency and explicitness for `MODULE.bazel`
-
Matthew Clarkson authored
Removes the need to have the MacOS SDK installed
-
Matthew Clarkson authored
As per the runfiles specification, runfile environment variables should be passed down to subprocesses.
-
Matthew Clarkson authored
Makes the execution of the symlink entirely hermetic. Previously, a direct symlink was used which required the APE binary to unpack and compile the embedded APE launcher. This requires a `cc` on the system. This patch shifts to use the already compiled APE launcher and a "indirect" symlink. The [indirect] symlink feature works by having a `.ape` extension to a symlink that points at the APE launcher. The launcher then removes the `.ape` extension and uses that as the path to the binary to launch. The crucial point is that assimilating an APE binary on MacOS is not natively launchable, unlike all other platforms: ```console $ cd /tmp $ curl -sfSL https://cosmo.zip/pub/cosmos/v/4.0.2/bin/assimilate > assimilate $ curl -sfSL https://cosmo.zip/pub/cosmos/v/4.0.2/bin/awk > awk $ chmod +x assimilate $ chmod +x awk $ ./ape # Already compiled from the patched `ape-m1.c` usage: ape PROG [ARGV1,ARGV2,...] ape - PROG [ARGV0,ARGV1,...] PROG.ape [ARGV1,ARGV2,...] actually portable executable loader silicon 1.10 copyrights 2023 justine alexandra roberts tunney https://justine.lol/ape.html $ ./ape ./awk awk version 20240311 $ ./awk xcode-select: note: No developer tools were found, requesting install. If developer tools are located at a non-default location on disk, use `sudo xcode-select --switch path/to/Xcode.app` to specify the Xcode that you wish to use for command line developer tools, and cancel the installation dialog. See `man xcode-select` for more details. $ ./ape ./assimilate -o awk.assimilated awk awk: ape macho dd command for arm64 not found; by convention ape executables are run on apple silicon only as elf binaries, which are loaded by the ape-m1.c ape loader program; thus consider passing the -ae flags to assimilate to arm64 elf (the host os uses macho natively) $ ./ape ./assimilate -ae -o awk.assimilated awk $ chmod +x awk.assimilated $ ./awk.assimilated zsh: exec format error: ./awk.assimilated $ ./ape ./awk.assimilated awk version 20240311 $ ln -s ape awk.ape # indirect symlink $ ./awk.ape awk version 20240311 $ ln -s awk.ape abc # symlink chain (does not work without the patch) awk version 20240311 ``` [indirect]: https://github.com/jart/cosmopolitan/blob/9f6bf6ea71e1385cc34dab0c492773f428d62869/ape/ape-m1.c#L1043-L1055
-
Matthew Clarkson authored
We are now using a feature of the APE launcher to use indirect symlinks. This allows the APE launcher to be symlinked with an `.ape` extension. The APE launcher will remove the `.ape` extension and then launch the binary at that path. This works really well under Bazel because it can handle creating the indirect and direct symlink for a APE binary. For example: ``` $ ls -l ape-arm64.macho awk $ ape-arm64.macho awk --version awk version 20240311 $ ln -s ape-arm64.macho awk.ape $ ./awk.ape # APE launcher removes the `.ape` and executes the `awk` APE binary awk version 20240311 ``` However, Bazel requires that a rule _must_ return files that it has declared and created. A common pattern is to simply symlink to a previously generated file and return that through the `DefaultInfo`. This breaks the indirect symlink because the downstream symlink can remove the `.ape` extension. Even if the symlink includes the `.ape` extension, unless it also generates a direct symlink for the APE launcher to find, it will fail to launch. This patches the APE launcher to walk symlink chains back to the final symlink. It adds a new function to the APE launcher binary: `WalkToIndirect`. The function uses two buffers to walk the symlinks until it finds a path that is not a symlink. The previous path _must_ be the indirect symlink. It needs to use an additional buffer to read the linkname. It will return zero if the new path can be stored in the provided buffer. If not, it will return the number of bytes to allocate. We use this functionality to store the path in the same character buffer used by the system provided environment. If there is not enough space, we then allocate on the stack and store it there. An alternative option is to build a launcher binary that uses runfiles to find both the APE launcher and binary. The preference was to avoid this to be resilient against subprocess calls that do not forward on runfiles information. It would also require a programming language that is hermetic, which would likely be `rules_go`. This is a heavy dependency for users just wanting to launch APE binaries on Apple Silicon. We are _already_ requiring the download of a hermetic C toolchain on Apple Silicon so it makes sense to reuse this download to provide the functionality needed for hermetic launching on Apple Silicon. This is an inperfect solution: all other platforms assimilate a single binary that can be launched on the system. Unfortunately, due to the limitations of the APE format on Apple Silicon, the APE launcher always needs to be in the loop as native binaries are assimilated to ELF. The downstream user now needs to be aware that on Apple Silicon, there are runfiles that need to be forwarded which limits the simple message that an `ape_assimilate` generates a native binary that can be launched. If APE learns how to generate launchable binaries on Apple Silicon, we can switch to `_direct` from `_indirect` and drop the `ape-m1.patch`.
-
Matthew Clarkson authored
Ensures we get `toolchain_utils@1.0.2` across the graph. We are not intending to upstream `toolchain_utils@1.0.1`.
-
Matthew Clarkson authored
-
25dc328a
-
af686ad8
-
862f8a0e
-
84ccedce
-
f52fcf7a
-
e3d2a156
-
d476ae42
-
b64271ed
-
dd4f2afa
-
Matthew Clarkson authored
-
Bot authored
## [1.0.1](https://git.gitlab.arm.com/bazel/ape/compare/v1.0.0...v1.0.1) (2025-02-12) ### Bug Fixes * bump to `toolchain_utils@1.0.2` for Bash 3.2 support ([90747dd9](https://git.gitlab.arm.com/bazel/ape/commit/90747dd993141fa5300498a0b993b662087edaf6)) * correct MacOS Apple Silicon launcher name ([45d9c1fe](https://git.gitlab.arm.com/bazel/ape/commit/45d9c1fe2ff27b380a1a2cff5b53deac966cde91)) * upgrade to `download_utils@1.0.1` ([9957f028](https://git.gitlab.arm.com/bazel/ape/commit/9957f0283242d332c8b80d3d5ae9693648a33d1e)) * use `ssize_t` for `readlink` return value ([dd4f2afa](https://git.gitlab.arm.com/bazel/ape/commit/dd4f2afaf8de0530a563b584b4abe2b2803c10b7)) * use `zig` target to build APE launcher ([d247cc5a](https://git.gitlab.arm.com/bazel/ape/commit/d247cc5af020d6fc32885c6680e2a32f372cb557)) * use indirect symlink on Apple Silicon ([c53415d6](https://git.gitlab.arm.com/bazel/ape/commit/c53415d60c45e7dc3ee732817d1abacbb297b0da)), closes [/github.com/jart/cosmopolitan/blob/9f6bf6ea71e1385cc34dab0c492773f428d62869/ape/ape-m1.c#L1043-L1055](https://git.gitlab.arm.com//github.com/jart/cosmopolitan/blob/9f6bf6ea71e1385cc34dab0c492773f428d62869/ape/ape-m1.c/issues/L1043-L1055) * walk symlinks on Apple Silicon ([e626a39d](https://git.gitlab.arm.com/bazel/ape/commit/e626a39d8b87954e390919b71ce98e072d3742a2)) * watch `zig`/`srcs` in `compile` repository rule ([48aa2786](https://git.gitlab.arm.com/bazel/ape/commit/48aa278682b56045e6bfa3ddcbbd1fde3ec33ea0))
Showing
- .bazelrc 2 additions, 2 deletions.bazelrc
- .gitlab-ci.yml 11 additions, 5 deletions.gitlab-ci.yml
- CHANGELOG.md 13 additions, 0 deletionsCHANGELOG.md
- MODULE.bazel 18 additions, 5 deletionsMODULE.bazel
- MODULE.bazel.lock 9 additions, 9 deletionsMODULE.bazel.lock
- ape-m1.patch 97 additions, 0 deletionsape-m1.patch
- ape/assimilate/rule.bzl 42 additions, 13 deletionsape/assimilate/rule.bzl
- ape/compile/BUILD.tmpl.bazel 1 addition, 1 deletionape/compile/BUILD.tmpl.bazel
- ape/compile/repository.bzl 30 additions, 1 deletionape/compile/repository.bzl
- ape/pe/repository.bzl 11 additions, 4 deletionsape/pe/repository.bzl
- e2e/.bazelrc 2 additions, 2 deletionse2e/.bazelrc
- e2e/MODULE.bazel 1 addition, 1 deletione2e/MODULE.bazel
- e2e/MODULE.bazel.lock 9 additions, 9 deletionse2e/MODULE.bazel.lock
- e2e/binary/__init__.py 2 additions, 1 deletione2e/binary/__init__.py
- e2e/binary/base32/BUILD.bazel 27 additions, 6 deletionse2e/binary/base32/BUILD.bazel
- e2e/binary/base32/digest.py 18 additions, 0 deletionse2e/binary/base32/digest.py
- e2e/binary/base32/expected.txt 1 addition, 0 deletionse2e/binary/base32/expected.txt
- e2e/binary/base32/fixture.txt 1 addition, 0 deletionse2e/binary/base32/fixture.txt
- e2e/binary/base64/BUILD.bazel 27 additions, 6 deletionse2e/binary/base64/BUILD.bazel
- e2e/binary/base64/digest.py 18 additions, 0 deletionse2e/binary/base64/digest.py
ape-m1.patch
0 → 100644
e2e/binary/base32/digest.py
0 → 100644
e2e/binary/base32/expected.txt
0 → 100644
e2e/binary/base32/fixture.txt
0 → 100644
e2e/binary/base64/digest.py
0 → 100644