Commit Graph

236 Commits

Author SHA1 Message Date
Thomas Petazzoni
9e7ef46065 tools: add fakeroot
SELinux support requires setting the appropriate SELinux security context
to files and directories, which needs to happen at build time in order
to support read-only root filesystem scenarios. In order to create these
security contexts, we will have to run some SELinux-specific tools on
the host machine, but that requires root access. This adds support for
fakeroot, which the build process will use to run the SELinux security
context creation and the image creation.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Apply to current master, and adjust commit message

Thomas' original work is available at
http://lists.infradead.org/pipermail/openwrt-devel/2019-November/025976.html.

Signed-off-by: W. Michael Petullo <mike@flyn.org>
[add rules.mk FAKEROOT variable]
Signed-off-by: Paul Spooren <mail@aparcar.org>
[update, fix macos build]
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-08-10 10:09:55 +02:00
Felix Fietkau
eb155f755a build: make prefix mapping of debug information optional
Remapping the local build path in debug information makes debugging
using ./scripts/remote-gdb harder, because files no longer refer to the full
path on the build host.

For local builds, debug information does not need to be reproducible,
since it will be stripped out of packages anyway.

For buildbot builds, it makes sense to keep debug information reproducible,
since the full path is not needed (nor desired) anywhere.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-08-06 12:36:24 +02:00
Roman Yeryomin
2ca084ccaa build: improve ccache support
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.

Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2020-07-11 15:19:53 +02:00
Sebastian Kemper
1f0063b58c rules.mk: remove "$(STAGING_DIR)/include"
"$(STAGING_DIR)/include" was carried over from buildroot-ng to OpenWrt
in commit 60c1f0f64d. buildroot has
dropped this directory a long time ago.

In OpenWrt the directory is still created by the PrepareStaging macro
and is part of the default TARGET_CPPFLAGS. But nothing at all installs
headers into this directory, nor should anything be installed under this
path.

Removing this directory from TARGET_CPPFLAGS will cut down the log noise
a bit. Not only will CPPFLAGS be shorter, there will be less warnings
set off by "-Wmissing-include-dirs" (or even failures when paired with
"-Werror"). After all the directory does not even _exist_ in the SDKs,
which are used on the build bots when building packages (see [1] and
[2]).

make[8]: Entering directory '/builder/shared-workdir/build/sdk/build_dir/target-aarch64_generic_musl/libmbim-1.20.0/src/common'
  CC       libmbim_common_la-mbim-common.lo
cc1: error: /builder/shared-workdir/build/sdk/staging_dir/target-aarch64_generic_musl/include: No such file or directory [-Werror=missing-include-dirs]
cc1: all warnings being treated as errors

[1] https://github.com/openwrt/packages/issues/10377
[2] https://github.com/openwrt/packages/pull/10378

Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Acked-by: Jo-Philipp Wich <jo@mein.io>
Acked-by: Rosen Penev <rosenp@gmail.com>
2019-11-02 20:51:56 +01:00
Paul Spooren
bd4d3cd101 toolchain,build: prefer -ffile-prefix-map for gcc-8+
-ffile-prefix-map=OLD=NEW is an alias for both -fdebug-prefix-map and
-fmacro-prefix-map and is available since GCC 8.

Co-Developed-by: Andre Heider <a.heider@gmail.com>
Signed-off-by: Andre Heider <a.heider@gmail.com>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[refactored into separate commit]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-09 09:13:44 +02:00
Martin Schiller
b204fcdb07 target/imagebuilder: use multi-thread support for xz compression
This speeds up the packing of the imagebuilder a lot:

 imagebuilder-T0.tar.xz real 0m25.199s user 2m45.967s sys 0m1.218s
 imagebuilder-T1.tar.xz real 2m02.543s user 2m02.418s sys 0m1.653s
 imagebuilder-T2.tar.xz real 1m03.684s user 1m59.931s sys 0m0.587s
 imagebuilder-T3.tar.xz real 0m48.033s user 2m02.904s sys 0m0.637s
 imagebuilder-T4.tar.xz real 0m38.963s user 2m15.521s sys 0m0.783s
 imagebuilder-T5.tar.xz real 0m37.994s user 2m21.461s sys 0m0.919s
 imagebuilder-T6.tar.xz real 0m39.524s user 2m48.115s sys 0m1.279s
 imagebuilder-T7.tar.xz real 0m34.061s user 2m45.097s sys 0m1.174s
 imagebuilder-T8.tar.xz real 0m27.286s user 2m55.449s sys 0m1.329s
 imagebuilder-T9.tar.xz real 0m25.205s user 2m44.894s sys 0m1.208s

To keep the output reproducible in any case, we enforce a minimum amount
of 2 threads.

Signed-off-by: Martin Schiller <ms@dev.tdt.de>
[refactored into reusable NPROC var, more verbose commit message]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-10-09 09:13:44 +02:00
Paul Spooren
e78c1baa9f rules: allow arbitrary log destination
Add option BUILD_LOG_DIR to menuconfig to change log destination.

The mix-up of *DIR* and *FOLDER* is confusing however.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-09-29 00:08:20 +02:00
Andy Walsh
94f6030170 librpc: remove package
* replaced with packages/libtirpc
* remove busybox options rarely used/deprecated
BUSYBOX_CONFIG_FEATURE_MOUNT_NFS
BUSYBOX_CONFIG_FEATURE_INETD_RPC

Signed-off-by: Andy Walsh <andy.walsh44+github@gmail.com>
2019-01-22 13:29:46 +01:00
Jo-Philipp Wich
1416b63dcb rules.mk: fix syntax error
Fix broken assignment operator added in a previous commit.

Fixes db73ec9f51 ("rules.mk: add INSTALL_SUID macro")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-11-29 12:33:15 +01:00
Jo-Philipp Wich
b0261ee5e9 rules.mk: add INSTALL_SUID macro
This is useful for packages that want to stage SUID executables.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-11-29 12:06:16 +01:00
Konstantin Demin
d3b43f49ac rules.mk: add ESED command
ESED is SED with extended regular expressions turned on.
Command line and usage are the same as for SED.

Signed-off-by: Konstantin Demin <rockdrilla@gmail.com>
2018-09-24 15:57:27 +01:00
Syrone Wong
713cee6463 toolchain/gcc: add config symbol to determine how to apply path remapping
Added boolean symbol for GCC 8 and higher, when we add newer GCC, we don't have
to modify rules.mk to keep things consistant.

Fixes: da9d760 ("rules.mk: replace iremap when using GCC 8")

Signed-off-by: Syrone Wong <wong.syrone@gmail.com>
2018-08-20 09:24:29 +02:00
Syrone Wong
da9d760ea1 rules.mk: replace iremap when using GCC 8
The original -iremap is replaced by -fmacro-prefix-map in GCC 8

Signed-off-by: Syrone Wong <wong.syrone@gmail.com>
2018-07-22 17:16:52 +02:00
Jo-Philipp Wich
681e825f8f build: include package directory in sha256sums when running on buildbot
In order to be able to better compare files to sync in the future, include
all BIN_DIR subdirectories in the checksum calculation.

To not break existing applications, restrict the recursive checksumming to
CONFIG_BUILDBOT for now.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-06-27 18:54:09 +02:00
Matthias Schiffer
2fbf669730
imagebuilder: reuse rootfs preparation from rootfs.mk
In addition to removing redundant code, this fixes various issues in
IB-generated images that have been fixed in prepare_rootfs before,
including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts
from FILES.

We also reuse the opkg macro and remove --force-... flags that have been
removed from rootfs.mk as well.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-03-07 09:59:08 +01:00
Alexandru Ardelean
cfdfa6d04e
rules.mk: drop include_mk build rule
The only users of this were the python packages
from the `packages` feed.
The 2 python interpreters would export some mk
files (e.g. python-package.mk) and then other
python packages would include it via this rule.

But there's a few things wrong with this approach,
most of them drawing from the fact that python host
needs to be built first, to export these mk files.

By now all uses of include_mk have been corrected
in the feeds and this can be removed.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
2018-01-26 12:55:14 +01:00
Felix Fietkau
bdc412941a build: remove use of STAGING_DIR_HOST/usr (fixes cmake build error on macOS)
This directory is never created, nor is it used anywhere. Using it in
HOST_LDFLAGS causes a linker warning to be emitted on macOS, which
messes with cmake configure tests.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-01-17 12:16:27 +01:00
Jo-Philipp Wich
72051f7036 rules.mk: export TMPDIR
Set TMPDIR to the same value as the existing TMP_DIR variable in order to
let gcc and various other utilities use the local temporary directory
instead of the system-wide one.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-12-12 17:44:01 +01:00
John Crispin
b957e455eb rukes.mk: this patch broken grub2 builds
Revert "rules.mk: add missing CPP definition"

This reverts commit 569f74ef49.

Signed-off-by: John Crispin <john@phrozen.org>
2017-09-01 10:17:22 +02:00
BangLang Huang
569f74ef49 rules.mk: add missing CPP definition
Signed-off-by: BangLang Huang <banglang.huang@foxmail.com>
2017-08-23 18:42:39 +02:00
Matthias Schiffer
5ef0854b11
rules.mk: make PKG_CONFIG_DEPENDS properly track string values
The confvar macro is adjusted to not only consider if a variable has a
value or not, but also the value itself. Instead of creating a string of
'y' and 'n' characters, all variable names and values are concatenated
and hashed.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-02-27 23:46:53 +01:00
Michal Sojka
202ae4cc6a build: Pass -iremap gcc option as a single argument
Passing -iremap argument separately causes problems with projects that
use scons and its ParseFlags function. Consider this SConscript
example:

    env = Environment()
    d = env.ParseFlags("-iremap one:two")

ParseFlags will interpret one:two as a file name and the returned dict
d will contain only "-iremap". When the -iremap is passed to the
compiler without an argument, compilation obviously fails.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
2017-02-09 14:49:34 +01:00
Felix Fietkau
d826af2cbb build: make <subdir>/install opt-in, use it for target/ only
Fixes buildbot errors on running make target/install or
toolchain/install

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-09 13:51:35 +01:00
Felix Fietkau
f55c29e4c7 build: remove libc version suffix from build/staging directories
Our supported libcs are ABI compatible across version upgrades

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-07 17:18:15 +01:00
Florian Fainelli
57657a7237 build: Suffix build directory with _$(LIBC) for external toolchains
For external toolchain, we also know the type of C library used, and the
toolchain triplet may not always be reflective of that, therefore make
$(TARGET_DIR_NAME) suffixed with _$(LIBC).

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-01-29 11:51:02 -08:00
Jo-Philipp Wich
7d57db4d9b build: introduce STAGING_DIR_IMAGE
Introduce a new location STAGING_DIR_IMAGE which is intended to be used by
bootloader iamges and similar image-related artifacts.

This directory is guaranteed to be persistent across kernel upgrades which
might involve a removal of KERNEL_BUILD_DIR and is guranteed to be bundled
with the image builder.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-27 16:53:31 +01:00
Matthias Schiffer
40d3401f3b
rules.mk: export STAGING_DIR_HOSTPKG
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-01-19 00:05:10 +01:00
Felix Fietkau
9228d1c066 build: introduce extra targets that contain only proper dependencies
This can be used to check if targets like prepare or compile are up to date

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau
6cf067d084 build: define common subdir targets in rules.mk
Reduce build system clutter and enable further rework

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:07 +01:00
Matthias Schiffer
3d71786154
build: move STAGING_DIR_HOSTPKG and BUILD_DIR_HOST back to a common directory for all targets
Using a single host package staging dir (and build dir) significantly speeds up
builds when multiple targets are built in succession, especially for large host
packages like NodeJS.

$(STAGING_DIR)/host is kept in addition to $(STAGING_DIR_HOSTPKG) in most
places; it is still used as destination for host files in Build/InstallDev.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-01-18 19:47:36 +01:00
Felix Fietkau
84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Felix Fietkau
ffb0181a87 build: add defaults for PKG_SOURCE, PKG_SOURCE_SUBDIR, PKG_VERSION
This makes it easier to unify versioning of git based package downloads.
PKG_SOURCE_DATE along with an 8-character abbreviation of the git hash
is used as PKG_VERSION, PKG_RELEASE should be used like normal packages.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-22 16:42:20 +01:00
Felix Fietkau
7a315b0b5d build: implement make check and make package/X/check
This is intended to be used for a wide array of package sanity checks.

The first check that is implemented is for the hash of downloaded files.
It checks:
  - Missing hash
  - Use of SHA256 instead of MD5
  - dl/<file> hash not matching hash in makefile
  - deprecated MD5SUM variable

The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-17 10:36:25 +01:00
Jo-Philipp Wich
d7bec5609c rules.mk: add STAGING_DIR_HOSTPKG variable
Add a STAGING_DIR_HOSTPKG variable which refers to $(STAGING_DIR)/host in order
to prepare support for relocating that directory in the future.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-11-01 12:11:14 +01:00
Jo-Philipp Wich
27854a0a84 build: add checksum target
Add a new "checksum" make target which generates an sha256sums file over the
image files produced in bin/targets/ and automatically call it during make
world after the package index generation.

The advantage of this new target is that it is guaranteed to run after the
images, the SDK and the ImageBuilder archives have been generated to ensure
that they all end up in the checksum file. Fixes FS#51.

Uses sed to postprocess the OpenSSL digest output into an sha256sum command
compatible format.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-01 18:11:21 +02:00
Jo-Philipp Wich
54fbe8afdd rules.mk: introduce new variable OUTPUT_DIR
Introduce a new variable OUTPUT_DIR which refers to the parent bin/ directory
without the architecture specific component.

Also construct the BIN_DIR variable based on OUTPUT_DIR now and add both target
and subtarget components to it to prevent different subtargets from
overwriting each others files in bin/.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-06 21:49:15 +02:00
Felix Fietkau
c8cbe7aba2 build: disable the use of -iremap for UML (#21851)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48718
2016-02-13 22:23:32 +00:00
Jo-Philipp Wich
064761d1db rules.mk: add TARGET_INIT_PATH toplevel variables
Add a new variable TARGET_INIT_PATH which holds the default $PATH variable
value configured in menuconfig.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 48676
2016-02-08 14:28:35 +00:00
Felix Fietkau
c5d74630c5 build: filter out -fno-plt for the kernel build, fixes #21712
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48515
2016-01-28 00:26:38 +00:00
Felix Fietkau
b737686b2c build: use -iremap feature to replace __FILE__ absolute path names in packages with ones relative to $(BUILD_DIR)
improves reproducible builds

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48489
2016-01-25 15:49:31 +00:00
Felix Fietkau
579e9620c6 rules.mk: make the locked template available even if flock has not been built yet (fall back to unlocked shell command)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48414
2016-01-20 21:31:38 +00:00
Felix Fietkau
c7308bc9f1 build: use a separate variable checking the subdir path (for host packages)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48410
2016-01-20 20:13:40 +00:00
Felix Fietkau
03bc8d94f0 host-build.mk: change default installation prefix of package/* host builds to staging_dir/target-*/host
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48401
2016-01-20 19:12:13 +00:00
Felix Fietkau
36502c5a6f build: move host build directories of package/* to build_dir/target-*/host/
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48400
2016-01-20 19:12:06 +00:00
Felix Fietkau
0628630664 build: add pure make tolower/toupper funtions that do not require shell calls
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48105
2016-01-03 20:57:53 +00:00
Felix Fietkau
f7939f5e74 gcc: remove version 4.6, it is no longer needed
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 47443
2015-11-10 21:10:53 +00:00
Felix Fietkau
940bc81bc2 build: add -Wno-error=unused-result to target cflags
Many glibc functions have __warn_unused_result__ in so many different
core functions, and failing the build for all of those simply does not
make any sense

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 47440
2015-11-10 12:04:09 +00:00
Felix Fietkau
279c8bee8d rules.mk: export STAGING_DIR_HOST to sub-makefiles and shells
Certains host packages (like quilt) need to know the previously used
install location at run-time. They currently use $STAGING_DIR, which
causes issues on cleaned environment, where $STAGING_DIR doesn't exist
yet.

Signed-off-by: Mathieu Olivari <mathieu@codeaurora.org>

SVN-Revision: 47409
2015-11-07 12:52:07 +00:00
Felix Fietkau
3bbb927728 build: Allow kernel modules to set build ID debug symbol
This change adds support for specifying a build ID for kernel modules.
This is done by setting PKG_BUILD_ID to a hexadecimal string, which will
then be passed to the kernel linker. In addition, when this flag is set,
the build ID debug symbol (.note.gnu.build-id) will not be stripped from
the kernel module. This symbol is exported in sysfs by the kernel (if
the kernel is compiled with CONFIG_KALLSYMS) and so can be used to
uniquely identify a version of a kernel module in a running kernel. This
is useful for keeping track of different versions of a module when doing
experiments and development.

Modules that specify the build ID will be ~100 bytes larger (depending
on the length of the build ID specified). There is no size difference
for kernel modules that do not set this variable.

Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>

SVN-Revision: 47290
2015-10-30 15:20:12 +00:00
Felix Fietkau
59bffdaa64 rules.mk: remove GCC 4.4 and GCC 4.5 conditional
gcc 4.4 was removed in r44957 gcc: remove 4.4.7 (only used by avr32)
gcc 4.5 was removed in r36149

Signed-off-by: Dirk Neukirchen <dirkneukirchen@web.de>

SVN-Revision: 46722
2015-08-25 07:45:42 +00:00