Commit Graph

30 Commits

Author SHA1 Message Date
Jonas Gorski
3210aa8e0a prereq: merge common cases in SetupHostCommand
Now that most cases do the same thing in SetupHostCommand, merge them
together into one. To allow moving the generic symlink check, invert the
check and let it check for relative links by matching on link targets
that do not start with a slash.

This then allows us to also drop the absolute link case, shortening the
case statement further.

This reorders the check to

* if it is not a symlink, do not change it
* if it is a symlink and it points to the found command, do not change it
* if it is a symlink with a relative path, do not change it
* else, update/replace it

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2023-08-26 14:00:36 +02:00
Jonas Gorski
c1ef10c8d8 prereq: make existing binary check work for sdk as well
To avoid replacing host built binaries with symlinks again, a check for
an appropriate stamp was added in 729909c07f ("prereq-build: do not
replace binaries with symlinks"). Unfortunately the stamp directory does
not exist in the SDK, so the fix was ineffective there.

This caused the packages builders to e.g. use the host tar again, which
in turn made the tarballs created different since it may lack
reproducibility fixes, or implement these differently, causing spurious
hash failures on source repository based packages.

Fix this by dropping the stamp dir check, and just check that the file
is usable.

Fixes: 729909c07f ("prereq-build: do not replace binaries with symlinks")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2023-08-26 14:00:36 +02:00
Michael Pratt
b890e2fbf9
prereq-build: replace relative symlinks only if broken
Some programs installed to staging_dir/host/bin
also install some symlinks to itself
for an alternative name.

Some of those new symlinks are overwriting
symlinks that were installed by prereq stage.

If prereq stage were to somehow be run again,
it should not be overwriting symlinks
that point to programs that are already built.

To filter that out, catch all symlinks
after first catching all symlinks
that have an absolute target
after all other cases in the case statement,
make sure it is not broken, and if so exit successfully.

Suggested-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-06-05 08:31:52 +02:00
Michael Pratt
729909c07f
prereq-build: do not replace binaries with symlinks
Some programs, like bash and patch, are checked by prereq stage
and have a symlink installed, but then is later built from source.

Now that the prereq-build checks are not successful
just by finding the file alone, it is possible for
a new symlink to overwrite the installed binary.

If a normal file is found in staging_dir/host/bin,
let the check look for the associated stampfile, and if found,
skip creation of a symlink and exit successfully.

Suggested-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-06-05 08:31:50 +02:00
Michael Pratt
9ffd6013b9
Revert "prereq: SetupHostCommand: fix wrong check result"
This reverts commit 7855378fcd.

The return "exit 1" was intentional and actually just
makes the symlink checks much more strict.

This new level of strictness added to the checks revealed
what was a confusing regression with prereq stage that
already existed but was not presenting itself
because of the simple way that checks used to be done before.

Either way, reverting to "exit 0" was a nice workaround
until the true root cause was discovered, so as to not interfere
with others' pull requests and builds in the meantime.

It turns out that this problem was the inconsistent value of $PATH
between different commands within the SetupHostCommand recipe,
now fixed in the parent commit, using the variable created
in the parent of the parent commit.

Ref: f75204036c ("prereq-build: allow host command symlinks to update")
Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-06-05 08:31:49 +02:00
Michael Pratt
665fe2f818
prereq-build: fix inconsistent value of $PATH
In the recipe SetupHostCommand for checking
and creating symlinks, $PATH was only overridden
for one of several commands.

This causes the symlinks to be included
in the paths to pick a program from
when the check was repeated, because
staging_dir/host/bin was included in $PATH,
but only *sometimes*.

When the check ran again, the command succeded
with a $PATH including the symlink,
(eval "$$$$$$$$cmd")
while the path to the program was evaluated
with a $PATH NOT including the symlink,
(bin=...)
causing the symlink to be relinked incorrectly,
instead of passing as exactly the same.

Coincidentally, this was only a problem
if the symlink targeted the alternative
program with a different name.

By overriding the value of $PATH at the invocation of Make,
it will apply to the entire environment of the checks.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-06-05 08:31:48 +02:00
Petr Štetiar
7855378fcd prereq: SetupHostCommand: fix wrong check result
Tony has reported, that CI tools job is failing for him in macOS
container due to prereq check failure for GNU `install` utility.

Michael diagnosed it and from his traces it was clear, that the issue is
caused by a wrong return value in the success check case, so lets fix it
accordingly.

Fixes: f75204036c ("prereq-build: allow host command symlinks to update")
Reported-by: Tony Ambardar <itugrok@yahoo.com>
Diagnosed-by: Michael Pratt <mcpratt@pm.me>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-23 17:49:57 +02:00
Michael Pratt
e2f9fa4204
prereq-build: remove python 2 cleanup recipe
This reverts commit 3b68fb57c9.

After refactoring build checks to update old symlinks,
and after a long time of no python 2 support,
this is no longer needed.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-05-21 19:39:35 +02:00
Michael Pratt
f75204036c
prereq-build: allow host command symlinks to update
This makes the prereq stage update the symlinks
installed into staging_dir/host/bin
by rearrainging the way they are verified.

Before, seeing or installing a symlink would result in
a successful exit code, and not installing a symlink
would result is a failed exit code. However,
that is not able to account for the difference
between existing good and bad links, or whether
the link would be the same if it was reinstalled,
because the check can match the program to a different path.

Instead, let a success exit code represent
identifying an existing symlink as exactly the same
as what would be installed if it did not exist,
and let a fail exit code represent
needing to install the symlink
or not having a match to the check criteria.

The failing exit code is caught by a new second attempt
for all of the check-* targets which will then indicate
to the user that there was an update by having a success
exit code when the check is run again and the link is the same.

When there is nothing to update, the checks will run only once.

This relies on the ls command to be POSIX-conformant with long format:
"path/to/link -> target/of/link"

Also, make sure the symlink is executable, not just a file,
and the directory only needs to be created once.

Fixes: #12610
Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-05-21 19:39:22 +02:00
Christian Marangi
35135842ca
include/prereq.mk: add RequireCHeader helper
Add RequireCHeader helper that will try to compile a fake c program with
the requested header included.

This is useful to check if a specific header is present in the system
without checking for the specific path.

This is a generilized version of the current ncurses test.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-09 21:25:06 +01:00
Ansuel Smith
88204bfa82 treewide: drop use of which
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-01-17 09:14:26 +01:00
Clemens Fruhwirth
1f5e722486 Revert "build: replace which with Bash command built-in"
This reverts commit c7aec47e5e.

The original commit replaces 'which' with 'command'. Sadly most of
them are not equivalent and for 'which -a', there is no easy
replacements that would not reimplement PATH parsing logic. Hence
revert. Keeping a dependency on which is absolutely fine.

Signed-off-by: Clemens Fruhwirth <clemens@endorphin.org>
2021-03-03 22:51:39 +01:00
Paul Spooren
a17b8eaa2e build: use SPDX license tags
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-05 14:54:47 +01:00
Petr Štetiar
c7aec47e5e build: replace which with Bash command built-in
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:

 $ python3 --version
 Python 3.9.1

 $ make
 /bin/sh: line 1: which: command not found
 /bin/sh: line 1: which: command not found
 /bin/sh: line 1: which: command not found
 ...
 Checking 'python3'... failed.
 ...

Fix this by switching to Bash builtin `command` which should provide
same functionality.

Fixes: FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2021-01-04 19:32:25 +01:00
Petr Štetiar
3b68fb57c9 build: cleanup possibly dangling Python 2 host symlink
When bumping buildroot to Python 3, we need to assure, that Python
symlink in staging bin directory points to Python >= 3.5 as well.

We can't rely completly just on SetupHostCommand as its executed only in
cases when the $(STAGING_DIR_HOST)/bin/python doesn't already exist, so
we need to remove it before running SetupHostCommand.

Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-26 08:09:16 +02:00
Jo-Philipp Wich
e9eb9393f4 include: extend SetupHostCommand macro to accept more arguments
Commit d6d3db0543 added more gcc version probes, exceeding the argument
limit of the SetupHostCommand macro, leading to failing GCC/LLVM tests
on OS X.

Extend the handled number of arguments to restore proper functionality.

Fixes FS#1470
Fixes d6d3db0543 ("build: Improve GCC version detection")

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-04-05 13:58:15 +02:00
Felix Fietkau
4d01d6ccb1 build: do not overwrite already existing host commands
This prevents a later prereq check from overwriting the installed tar
binary.

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 46982
2015-09-16 12:38:16 +00:00
Luka Perkov
7002f6326e include: remove trailing whitespaces
Signed-off-by: Luka Perkov <luka@openwrt.org>

SVN-Revision: 45127
2015-03-29 07:29:18 +00:00
Felix Fietkau
834417623d build: accept apple llvm as fallback for gcc/g++
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44298
2015-02-07 10:07:31 +00:00
Felix Fietkau
64c4ee703a include/prereq.mk: fix echo commands with simple /bin/sh
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44297
2015-02-07 10:07:05 +00:00
Jo-Philipp Wich
9aca57b6ae include: don't rely on perl for printing prereq errors
Perl might not be present while checking prereqs, e.g. on a fresh
FreeBSD install.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 44276
2015-02-05 16:57:46 +00:00
Jo-Philipp Wich
94e528f159 include: add TestHostCommand and SetupHostCommand macros
Those macros allow testing various commands on the host system, also process the
prereq tests in the order they're declared.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 44269
2015-02-05 16:57:02 +00:00
Felix Fietkau
d99a680230 prereq.mk: only define .NOTPARALLEL when processing prereq checks
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 42877
2014-10-12 15:00:41 +00:00
Felix Fietkau
800ef0f56c some minor fixes, cleanups, package build abstraction
SVN-Revision: 9051
2007-09-28 01:23:56 +00:00
Florian Fainelli
0860346ff0 Add a requireheader for squid, should fix it (#2205)
SVN-Revision: 8404
2007-08-12 11:53:23 +00:00
Felix Fietkau
d7f2cb1642 next round of cleanup, convert target/ - make -j works now ;)
SVN-Revision: 8242
2007-07-30 21:14:08 +00:00
Mike Baker
dc4f082aea print package name when prereq fails
SVN-Revision: 6236
2007-01-31 20:17:28 +00:00
Felix Fietkau
d8cc1426e4 allow newlines in prereq error messages
SVN-Revision: 6147
2007-01-20 18:45:06 +00:00
Felix Fietkau
cf4e7af392 make target/linux/* directories self-contained, use the selected kernel version for the toolchain, autogenerate menuconfig and makefile code for target selection
SVN-Revision: 5498
2006-11-11 23:11:02 +00:00
Felix Fietkau
60c1f0f64d finally move buildroot-ng to trunk 2016-03-20 17:29:15 +01:00