Commit Graph

137 Commits

Author SHA1 Message Date
Sandro Jäckel
86ca7199df
build: fix pkg-config detection when inside of a nix-shell
The output of command_all when inside a nix-shell looks like the below
where /usr does not match:

 ➜ scripts/command_all.sh pkg-config
/nix/store/ifr6srqgpvygd5vp14748d109ri31isv-pkg-config-wrapper-0.29.2/bin/pkg-config

Signed-off-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
2023-11-02 20:26:32 +01:00
Michael Pratt
d87a8aa148
treewide: add ORIG_PATH variable
Add a variable that stores the original value of $PATH
in the host system's shell, before Make alters it.

This can be useful for when it is necessary
to ignore symlinks and programs made by the build system.

Define this new variable before all instances of
'export PATH:=' or similar.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2023-06-05 08:31:47 +02:00
Christian Marangi
2a3283643c
treewide: derive host and hostpkg path from STAGING_DIR
STAGING_DIR may be provided from command line. We currently hardcoded
STAGING_DIR_HOST and STAGING_DIR_HOSTPKG to the default location but we
currently have some relocatable patch that derive the path from
STAGING_DIR.

Fix this and correctly derive STAGING_DIR_HOST and STAGING_DIR_HOSTPKG
from STAGING_DIR.

The intention is to fix inconsistency from the relocatable patch and the
use of STAGING_DIR_HOST that is always hardcoded.
This with a wrong configuration may end up in broken state with some
host tools expecing a PATH from STAGING_DIR and others using library
from the default staging_dir/host path.

To save downstream project the original implementation is saved while
fixing the inconsistency between patch and .mk.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-09 21:33:20 +01:00
Nick Hainke
a29d3bc48c
Makefile: fix stray \ warnings with grep-3.8
We simply grep for "/usr". So no need for "-E" or "\/". Furthermore, in
the new grep versions this creates warnings.

As written in the grep-3.8 announcement:
  Regular expressions with stray backslashes now cause warnings, as
  their unspecified behavior can lead to unexpected results.
  For example, '\a' and 'a' are not always equivalent
  <https://bugs.gnu.org/39678>.

Fixes warnings in the form of:
  grep: warning: stray \ before /

Signed-off-by: Nick Hainke <vincent@systemli.org>
2022-09-27 16:45:08 +02:00
Rosen Penev
22b7bd6b13
Makefile: replace head call with grep's -m
head is not necessary here.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-10 15:42:36 +02:00
Huangbin Zhan
aeaa816cd2 build: don't remove BUILD_LOG_DIR in _clean
targetclean should not remove BUILD_LOG

Fixes: db34b93331 (add a version that can be bumped to force toolchain/target rebuild)
Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
2022-04-30 23:56:43 +02:00
Ansuel Smith
88204bfa82 treewide: drop use of which
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-01-17 09:14:26 +01:00
Felix Fietkau
db34b93331 toolchain: add a version that can be bumped to force toolchain/target rebuild
This can be used to ensure that the compiled code is up to date, when
something important changes in the toolchain.
A recent example of this is the gcc 11 fix for a code miscompilation issue

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-16 22:02:45 +01:00
Clemens Fruhwirth
1f5e722486 Revert "build: replace which with Bash command built-in"
This reverts commit c7aec47e5e.

The original commit replaces 'which' with 'command'. Sadly most of
them are not equivalent and for 'which -a', there is no easy
replacements that would not reimplement PATH parsing logic. Hence
revert. Keeping a dependency on which is absolutely fine.

Signed-off-by: Clemens Fruhwirth <clemens@endorphin.org>
2021-03-03 22:51:39 +01:00
Daniel Golle
1b6209203d
Revert "build: use config.site generated by autoconf-lean, drop hardcoded sitefiles"
This reverts commit f439e29130.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-02 11:35:45 +00:00
Felix Fietkau
f439e29130 build: use config.site generated by autoconf-lean, drop hardcoded sitefiles
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-28 00:09:09 +00:00
Paul Spooren
a17b8eaa2e build: use SPDX license tags
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-05 14:54:47 +01:00
Sven Wegener
cd5f66652b build: use ccache -C for cleaning the cache
This keeps the configuration, like the size of the cache, and the
statistics intact. Move the removal of the cache directory to the
distclean target, but only delete the .ccache directory inside of our
build tree, as we should not mess with a user-configured external ccache
directory this destructively.

Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
2021-01-06 15:31:18 -10:00
Petr Štetiar
c7aec47e5e build: replace which with Bash command built-in
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:

 $ python3 --version
 Python 3.9.1

 $ make
 /bin/sh: line 1: which: command not found
 /bin/sh: line 1: which: command not found
 /bin/sh: line 1: which: command not found
 ...
 Checking 'python3'... failed.
 ...

Fix this by switching to Bash builtin `command` which should provide
same functionality.

Fixes: FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2021-01-04 19:32:25 +01:00
Sven Wegener
01653fe3c7 build: use CCACHE_DIR for ccache directory
It is being set by rules.mk with the same expression.

Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
2021-01-01 13:31:11 -10:00
Roman Yeryomin
2ca084ccaa build: improve ccache support
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.

Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2020-07-11 15:19:53 +02:00
Paul Spooren
07449f692c build: refactor JSON info files to profiles.json
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").

They are useful for firmware wizards and script checking for
reproducibility.

Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.

This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.

Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.

Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.

As before, this creation is enabled by default only if `BUILDBOT` is set.

Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-04-03 12:17:45 +02:00
Kevin Darbyshire-Bryant
0c4d91f121 build: clean menuconfig utility as part of dirclean
When sharing a common build directory between different build platforms
eg. macos v docker based linux v virtual machine, a 'make dirclean'
isn't quite enough to clean all the platform related binaries.  The
'conf' and 'mconf' aka 'make menuconfig/defconfig & friends' utilities
remain.

Clean those as part of 'dirclean' so they get rebuilt for the current
platform on the next 'make'

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2020-03-10 20:44:30 +00:00
Paul Spooren
6caf437652 build: add buildinfo as single Makefile target
The prepare target was added some 11 years ago to build tools and
toolchain and was recently extended to  create buildinfo files for
reproducibility, meaning {feeds,version,config}.buildinfo.

As the buildbot workflow is more complex than the single prepare (kmod
feed insertion), prepare is only used to create those buildinfo files.
Running prepare however runs `target/compile` as well, taking time even
everything is already compiled.

Splitting this allows the buildbot to run only the `buildinfo` target
while others can still use the convenience feature `prepare`.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-17 14:01:33 +02:00
Petr Štetiar
965f341aa9 build: fix host menu config targets using ncurses
On a recent Gentoo Linux installation, invoking `make menuconfig`, `make
kernel_menuconfig` or `make kernel_nconfig` in the build system fails,
whereas for example `make menuconfig` in the kernel tree alone works as
expected.

This is happening because STAGING_PREFIX is not defined when kernel's
{menu,n}config target calls pkg-config from the toolchain/host and thus
pkg-config returns an empty value, and the fallback values in the kernel
config script are applied but those are off and the linking fails.

Solution is to use system's pkg-config for all ncurses based menu config
targets in order to provide proper compiler/linker flags.

Ref: FS#2423
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Tested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-09-25 12:50:24 +02:00
Paul Spooren
454021581f build: add buildinfo files for reproducibility
generate feeds.buildinfo and version.buildinfo in build dir after
containing the feed revisions (via ./scripts/feeds list -sf) as well as
the current revision of buildroot (via ./scripts/getver.sh).

With this information it should be possible to reproduce any build,
especially the release builds.

Usage would be to move feeds.buildinfo to feeds.conf and git checkout the
revision hash of version.buildinfo.

Content of feeds.buildinfo would look similar to this:

    src-git routing https://git.openwrt.org/feed/routing.git^bf475d6
    src-git telephony https://git.openwrt.org/feed/telephony.git^470eb8e
    ...

Content of version.buildinfo would look similar to this:

    r10203+1-c12bd3a21b

Without the exact feed revision it is not possible to determine
installed package versions.

Also rename config.seed to config.buildinfo to follow the recommended
style of https://reproducible-builds.org/docs/recording/

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-08-13 10:40:36 +02:00
Felix Fietkau
7ec092e641 Revert faulty tree push
Revert "mac80211: add new minstrel_ht patches to improve probing on mt76x2" (9861050b85)
Revert "kernel: use bulk free in kfree_skb_list to improve performance" (98b654de2e)
Revert "ramips: add preliminary support for WIO ONE" (085141dc5b)
Revert "ramips: add preliminary support for SGE AP-MTKH7-0006 developer board" (b1db6d0539)
Revert "build: use config.site generated by autoconf-lean, drop hardcoded sitefiles" (363ce4329d)
Revert "toolchain: add autoconf-lean" (fdb30eed03)
Revert "build: allow overriding the filename on the remote server when downloading" (6fa0e07758)

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-08-12 12:27:06 +02:00
Felix Fietkau
363ce4329d build: use config.site generated by autoconf-lean, drop hardcoded sitefiles
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2019-08-12 11:43:39 +02:00
Petr Štetiar
12610a5620 Revert "build: allow simple build system customization with local.mk"
This reverts commit fe928c7a81 as it seems
to introduce build breakage when local.mk doesn't exist.

 make --trace
 toplevel.mk:216: target 'local.mk' does not exist

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-25 09:48:33 +02:00
Petr Štetiar
fe928c7a81 build: allow simple build system customization with local.mk
I tend to automate most of the boring and repetitive tasks like firmware
flashing, build config reconfiguration etc., so I always end up adding
`include local.mk` as a last line in my Makefile, where local.mk is
usually symlink to some other place, just to not accidentally delete it
during `git clean`.

Carrying this single uncommited modification along in the development
process is quite PITA, because it's causing problems during Git
workflow, while rebasing etc.

I hope, that I'm not alone using similar workflow, so I believe, that
this modification might be useful for others as well.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-07-25 07:48:02 +02:00
Thomas Langer
41faf52b0f build: Unset CDPATH to avoid problems
In some places the output of commands, which include "cd" are used.
In case of CDPATH the new path is printed, which might not be expected.
Disable the variable to avoid these problem.

When CDPATH was set by the user to some value like "export CDPATH=."
the git checkout done by the build system did not work anymore, the
git cloning aborted with such an error message for example:
....
Packing checkout...
tar: /disk/fs1/tmp2/mehrtens/pon-ugw/ugw-haps/openwrt/tmp/dl/ppa-drv-1.0\n@1534240258: Cannot stat: No such file or directory
tar: Date sample file not found
Try 'tar --help' or 'tar --usage' for more information.
.....

To avoid this, this patch makes the build system unset CDPATH inside
the build system, so the build system will still work even when the
user set this variable in his local environment.

Signed-off-by: Thomas Langer <thomas.langer@intel.com>
Signed-off-by: Hauke Mehrtens <hauke.mehrtens@intel.com>
Acked-by: Hans Dedecker <dedeckeh@gmail.com>
2018-08-25 11:10:28 +02:00
Jo-Philipp Wich
681e825f8f build: include package directory in sha256sums when running on buildbot
In order to be able to better compare files to sync in the future, include
all BIN_DIR subdirectories in the checksum calculation.

To not break existing applications, restrict the recursive checksumming to
CONFIG_BUILDBOT for now.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-06-27 18:54:09 +02:00
Zoltan HERPAI
34bbbbf9c3 merge: base: update base-files and basic config
Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
2017-12-08 19:41:18 +01:00
Vitaly Chekryzhev
3ceb0fa7c5 build: prepare config.seed before package compilation
Signed-off-by: Vitaly Chekryzhev <13hakta@gmail.com>
2017-03-18 12:08:04 +01:00
Felix Fietkau
9467ce42da build: get rid of host.mk
Defined required host related variables in toplevel.mk instead

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Felix Fietkau
664918d891 Revert "build: always run package/cleanup before package/compile"
This reverts commit 2990a21058.
This introduces a race condition, let's fix this in buildbot instead.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-22 13:47:40 +01:00
Felix Fietkau
2990a21058 build: always run package/cleanup before package/compile
Remove unnecessary stampfile indirection
Fixes an issue with the command sequence used by buildbot

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-22 13:31:54 +01:00
Felix Fietkau
131db366a3 build: remove separate /install step for host builds
Reduces the number of recursive make invocations

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:09 +01:00
Matthias Schiffer
3d71786154
build: move STAGING_DIR_HOSTPKG and BUILD_DIR_HOST back to a common directory for all targets
Using a single host package staging dir (and build dir) significantly speeds up
builds when multiple targets are built in succession, especially for large host
packages like NodeJS.

$(STAGING_DIR)/host is kept in addition to $(STAGING_DIR_HOSTPKG) in most
places; it is still used as destination for host files in Build/InstallDev.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-01-18 19:47:36 +01:00
Hannu Nyman
c1b12aa838 Makefile: ensure that BIN_DIR exists for diffconfig
Ensure that BIN_DIR exists when the diffconfig target needs it.
Otherwise 'make diffconfig' fails after 'make clean'

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
2017-01-08 18:50:00 +01:00
Jo-Philipp Wich
2a72a916ab build: add diffconfig target
Add a "diffconfig" build target which stores the output of
"scripts/diffconfig.sh" as "config.seed" in the image output directory and
invoke that target by default.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-04 11:03:53 +01:00
Felix Fietkau
7a315b0b5d build: implement make check and make package/X/check
This is intended to be used for a wide array of package sanity checks.

The first check that is implemented is for the hash of downloaded files.
It checks:
  - Missing hash
  - Use of SHA256 instead of MD5
  - dl/<file> hash not matching hash in makefile
  - deprecated MD5SUM variable

The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:25 +01:00
Felix Fietkau
f2de33b2c9 build: move merged package directory from bin/ to staging_dir
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-03 12:22:18 +02:00
Jo-Philipp Wich
27854a0a84 build: add checksum target
Add a new "checksum" make target which generates an sha256sums file over the
image files produced in bin/targets/ and automatically call it during make
world after the package index generation.

The advantage of this new target is that it is guaranteed to run after the
images, the SDK and the ImageBuilder archives have been generated to ensure
that they all end up in the checksum file. Fixes FS#51.

Uses sed to postprocess the OpenSSL digest output into an sha256sum command
compatible format.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-01 18:11:21 +02:00
Felix Fietkau
30d955a7a5 build: fix make clean, delete package directories for selected arch
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-11 10:02:36 +02:00
John Crispin
fa69553900 branding: add LEDE branding
Signed-off-by: John Crispin <blogic@openwrt.org>
2016-03-24 22:40:13 +01:00
Felix Fietkau
868d5f9ae9 Makefile: set timezone to UTC
This is necessary for reproducible image builds.

Signed-off-by: bryan newbold <bnewbold@robocracy.org>

SVN-Revision: 48538
2016-01-28 22:42:18 +00:00
Felix Fietkau
b20a2b4890 Makefile: move the cleaning of staging_dir/target* from dirclean to clean
Currently "make clean" only clears the build_dir/target*, but leaves
staging_dir/target* intact. "make clean" should also clean the
staging_dir/target* directories, as in the current situation some old
packages or libraries may be linked into the firmware from staging_dir
despite a "make clean".

The patch reorganises clean / dirclean functionality so that
* "make clean" also clears the staging_dir/target* in addition to
build_dir/target*.
* "make dirclean" clears toolchain and host(=tools) directories from both
build_dir and staging_dir

signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 45973
2015-06-14 17:47:16 +00:00
John Crispin
9a96820d3d Makefile: remove non-existent STAGING_DIR_TOOLCHAIN from dirclean
Makefile: remove non-existent STAGING_DIR_TOOLCHAIN from dirclean

Openwrt's top level Makefile uses STAGING_DIR_TOOLCHAIN in the make dirclean
statement.
https://dev.openwrt.org/browser/trunk/Makefile#L55
    rm -rf $(STAGING_DIR) $(STAGING_DIR_HOST) $(STAGING_DIR_TOOLCHAIN)
$(TOOLCHAIN_DIR) $(BUILD_DIR_HOST) $(BUILD_DIR_TOOLCHAIN)

As far as I can determine, no such variable has been defined. I made a search
in Openwrt source repository and the one line in Makefile's dirclean command
is the only place where that variable exists.

The item has been introduced to Makefile by r8362, but even at that time
neither Makefile nor rules.mk defined such a variable. Most likely the goal
has been to set both staging_dir/toolchain and build_dir/toolchain to be
cleaned, but one of the variables has been erroneous. The correct variable
for build_dir/toolchain has been then added by r13494.

References:
https://dev.openwrt.org/changeset/8362/
https://dev.openwrt.org/browser/trunk/Makefile?rev=8362
https://dev.openwrt.org/browser/trunk/rules.mk?rev=8362
https://lists.openwrt.org/pipermail/openwrt-devel/2007-August/001159.html
https://dev.openwrt.org/changeset/13494

In current code,
    TOOLCHAIN_DIR = $(TOPDIR)/staging_dir/$(TOOLCHAIN_DIR_NAME)
    BUILD_DIR_TOOLCHAIN = $(TOPDIR)/build_dir/$(TOOLCHAIN_DIR_NAME)
so the item STAGING_DIR_TOOLCHAIN in the rm command is unnecessary.

signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 45736
2015-05-23 15:28:42 +00:00
Felix Fietkau
5868331214 build: don't call prereq for any package/symlinks rules
Most of the time, we want to make sure OpenWrt has been configured and
setup before start running make. However, in case of package/symlinks,
forcing prereq as a dependency creates multiple issues:
*when executed on a clean workspace, it will prompt for user input
 and open a menuconfig window before executing the feeds command
*the only way around that is to provide a .config. However, the "prereq"
 target would then run a "make defconfig", which will remove all the
 packages in the .config but from external feeds, as feeds have not been
 installed yet.

The only way to currently work around this, is to generate a fake config
by running "make defconfig", then "make package/symlinks", copy the real
config (which at this point disregards the previously generated config),
and run make defconfig again. Something like this:

make defconfig
make package/symlinks
cp real.config .config
make defconfig

This change is removing the need for the first defconfig, making the
process more logical for OpenWrt users using the package/symlinks target.

Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>

SVN-Revision: 45657
2015-05-10 11:17:29 +00:00
Felix Fietkau
74ca7e9af3 build: clean up sitefiles, make them per-arch
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45090
2015-03-28 13:38:34 +00:00
Felix Fietkau
4fde9deed1 Revert "build: use ONESHELL to speed up scanning and the toplevel makefile"
This seems to cause issues on some build hosts, and it is not that
important.

SVN-Revision: 43025
2014-10-22 10:47:26 +00:00
Felix Fietkau
c497990834 build: use ONESHELL to speed up scanning and the toplevel makefile
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43021
2014-10-22 08:57:40 +00:00
Felix Fietkau
bb959c0f3e build: remove duplicate variable definitions
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39981
2014-03-21 15:54:11 +00:00
Felix Fietkau
c233a088f0 build: it is no longer necessary to call make target/linux/clean as part of make clean
SVN-Revision: 34451
2012-12-02 15:26:28 +00:00