Commit Graph

138 Commits

Author SHA1 Message Date
Tony Butler
4ab4b9ea81
build: fix incorrect initramfs gzip compression
Requires: tools/libdeflate

fix consistency of executable to use
`$(STAGING_DIR_HOST)/bin/libdeflate-gzip`, and not system-installed ones
from the usual environment `PATH`;
this affects option `CONFIG_KERNEL_INITRAMFS_COMPRESSION_GZIP`

this may have worked in the past but only via side effect of having the
binaries on the host system (and whatever unpredictable version or
patchset those might be), and did not use the improved but totally
compatible libdeflate-gzip

Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-04-18 13:57:48 +02:00
Tony Butler
69bc620180
build: fix incorrect initramfs bzip2 compression
Requires: tools/bzip2

fix consistency of executable to use `$(STAGING_DIR_HOST)/bin/bzip2`, and
not system-installed ones from the usual environment `PATH`;
this affects option `CONFIG_KERNEL_INITRAMFS_COMPRESSION_BZIP2`

this may have worked in the past but only via side effect of having the
binaries on the host system (and whatever unpredictable version or
patchset those might be)

Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Signed-off-by: Tony Butler <spudz76@gmail.com>
2023-04-18 13:57:47 +02:00
Rosen Penev
11ad0c8355
include: use libdeflate's gzip to decompress
libdeflate decompresses much faster than gzip.

Example:

~/d/openwrt> time gzip -dc dl/cmake-3.25.1.tar.gz > /dev/null

________________________________________________________
Executed in    1.01 secs      fish           external
   usr time  912.61 millis    1.67 millis  910.94 millis
   sys time   32.21 millis    0.25 millis   31.96 millis

~/d/openwrt> time libdeflate-gzip -dc dl/cmake-3.25.1.tar.gz > /dev/null

________________________________________________________
Executed in  523.04 millis    fish           external
   usr time  415.48 millis    1.07 millis  414.41 millis
   sys time  107.74 millis    0.15 millis  107.59 millis

~/d/openwrt> time gzip -dc dl/tessdata-4.1.0.tar.gz > /dev/null

________________________________________________________
Executed in    8.99 secs    fish           external
   usr time    8.90 secs  530.00 micros    8.90 secs
   sys time    0.07 secs   63.00 micros    0.07 secs

~/d/openwrt> time libdeflate-gzip -dc dl/tessdata-4.1.0.tar.gz > /dev/null

________________________________________________________
Executed in    2.74 secs    fish           external
   usr time    2.38 secs  537.00 micros    2.38 secs
   sys time    0.35 secs   66.00 micros    0.35 secs

Place libdeflate into tools-core as it is needed to decompress other
archives.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-17 21:40:18 +01:00
Tony Butler
4a444e576f build: fix incomplete initramfs compression options
Requires: tools/lz4, tools/lzop

complete the wiring so that these options work:
* `CONFIG_KERNEL_INITRAMFS_COMPRESSION_LZO`
* `CONFIG_KERNEL_INITRAMFS_COMPRESSION_LZ4`

Signed-off-by: Tony Butler <spudz76@gmail.com>
[remove blocking dependencies for separate ramdisk, fix lzop options]
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2023-01-17 00:16:35 +00:00
Christian Lamparter
1ce2691bf3 uml: headers are not exportable
When building UML this showed up as of late (2022-09-01):
| Makefile:1335: *** Headers not exportable for UML.  Stop.

This message popped up because of an upstream patch that
fixed this error not showing up sooner. For more information see:
<https://lore.kernel.org/lkml/20220901011252.550830-1-masahiroy@kernel.org/T/>

Now, I don't think we can just skip the kernel headers.
Some userspace applications are going to need these. So,
The next best alternative I can think of, is to go with
the headers by the host arch the UML is compiled for.
(Technically, ARCH= should be safe for all other targets
as well... But let's not poke a hornet's nest)

Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2022-09-24 23:52:14 +02:00
Daniel Golle
54bcf586b0
kernel: set SOURCE_DATE_EPOCH for initramfs root dir
Make sure the timestamp of the root directory of the initramfs is set
to SOURCE_DATE_EPOCH as well.

Fixes: 29d7461d11 ("kernel: set options to make external initramfs reproducible")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-04-06 12:20:26 +01:00
Daniel Golle
fc6a83e63b
kernel: fix initramfs reproducibility
Make sure xz uses at least 2 threads so compression always runs in
multi-threaded mode as the resulting file in single-threaded mode
differs.

Fixes: 29d7461d11 ("kernel: set options to make external initramfs reproducible")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-03-29 14:00:42 +01:00
Daniel Golle
29d7461d11 kernel: set options to make external initramfs reproducible
Run cpio as well as compressors in such ways that they are generating
reproducible output.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-03-24 03:28:50 +00:00
Daniel Golle
0a4f5d06c2
image: fix CONFIG_EXTERNAL_CPIO handling
CONFIG_EXTERNAL_CPIO is a string variable, hence testing for 'y'
doesn't make much sense here.

Fixes: 330bd380e8 ("image: allow building FIT and uImage with ramdisk")
Reported-by: Huangbin Zhan <zhanhb88@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-11-28 20:15:10 +00:00
Eneas U de Queiroz
6b8b5e7248 build: kernel: avoid unnecessary package rebuilds
The $(LINUX_DIR)/.config timesptamp changes between runs of
make target/compile and make target/install (which builds the image).

Kernel-dependent packages and out of tree modules are built in between
those runs, and they check the .config timestamp to decide if they need
to be rebuilt.

Save the target/compile .config to use its timestamp if the file does
not change between runs.  That way the subsequent kernel packages are
not unnecessarily rebuilt when you run 'make' back to back.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2021-11-01 20:18:55 +01:00
Sebastian Kemper
54070a1f5c build: fix regression for kernels < 5.10
This fixes a regression introduced with commit
5ed1e5140a ("build: build kernel image
before building modules/packages").

Before this commit the make target would always include "modules",
resulting in a MODPOST and a complete Module.symvers file. Since this
commit a MODPOST of the kernel modules is not guaranteed for kernels <
5.10. This results in some broken SDKs in which external packages that
depend on exported symbols from kernel modules fail to compile.

Adding "modules" back to the calls to the CompileImage defines fixes the
regression. For kernels > 5.10 this is not needed, but it doesn't cause
any harm either.

Tested with kernels 5.4.x and 5.10.x.

Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
2021-05-25 19:46:13 +02:00
Matt Merhar
cc76e34c10 kernel-defaults: fix external kernel build when user_headers is missing
Use an 'if' so the absence of $(LINUX_DIR)/user_headers doesn't make the
line evaluate to false and cause the build to fail.

Signed-off-by: Matt Merhar <mattmerhar@protonmail.com>
2021-05-23 15:11:38 +02:00
Leonardo Mörlein
b993b68b6c build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
2021-05-13 15:13:15 +02:00
Felix Fietkau
785ab2b62c build: use numeric uid/gid on cpio calls
There are systems that don't have the 'root' group, so don't rely on host
specific user/group names

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-03-03 16:09:59 +01:00
Daniel Golle
ebcb4f1d0a
treewide: fix spelling 'seperate' -> 'separate'
This popular spelling mistake was also introduced by myself lately.
Fix it everywhere.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-28 23:59:21 +00:00
David Bauer
dc5328e7e9 include: use cpio from staging dir
As we built our own CPIO now, use this version instead of whatever the
host may or may not provide.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-28 00:09:09 +00:00
Daniel Golle
330bd380e8 image: allow building FIT and uImage with ramdisk
Instead of embedding the initrd cpio archive into the kernel, allow
for having an external ramdisk added to the FIT or uImage.
This is useful to overcome kernel size limitations present in many
stock bootloaders, as the ramdisk is then loaded seperately and doesn't
add to the kernel size. Hence we can have larger ramdisks to host ie.
installers with all binaries to flash included (or a web-based
firmware selector).
In terms of performance and total size the differences are neglectible.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-02-24 01:35:20 +00:00
Felix Fietkau
5ea33837f8 build: fix build with CONFIG_STRIP_KERNEL_EXPORTS
Only use symtab.h on the final kernel link

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-17 13:49:43 +01:00
Felix Fietkau
299b855418 build: make zstd initramfs selectable
fix typo in kernel initramfs zstd compression option

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-16 20:02:09 +01:00
Felix Fietkau
5ed1e5140a build: build kernel image before building modules/packages
This is needed for linux 5.10, where modules.builtin is generated from
vmlinux.o

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-02-16 20:00:41 +01:00
Paul Spooren
a17b8eaa2e build: use SPDX license tags
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-05 14:54:47 +01:00
Florian Fainelli
0f8fd1d0bf kernel-defaults: Manage the enabling/disabling of ZSTD
Linux 5.9 introduces support for ZSTD ramdisk and initrd compression,
make sure we enable/disable the relevant options when building an
initramfs enabled kernel.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2021-01-09 14:49:48 -08:00
Florian Fainelli
09760cbb3b kernel-defaults: Delete external source tree user_headers
When we use an external kernel tree which may not have been fully
cleaned, there may be user_headers left which do not match the target
architecture, leading to build failures for packages that do an explicit
inclusion of user_headers (such as iproute2 or iptables). Make sure we
delete them while preparing the directory.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2021-01-09 14:49:48 -08:00
Felix Fietkau
ef11309c22 build: make testing kernel decompression more portable
On non-GNU systems, zcat often does not handle gzip decompression.
Use gzip -dc like the regular unpack command

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-12-05 12:06:30 +01:00
Paul Spooren
4791afa734 kernel-defaults: ensure SOURCE_DATE_EPOCH on /init
This is a follow up of 8cb13f4e6d which sets the right timestamp for the
/init file in initramfs. The previous patch doesn't cover it as the
files appear to come from a later step during compilation.

CC: Alexander Couzens <lynxis@fe80.eu>

Signed-off-by: Paul Spooren <mail@aparcar.org>
2019-10-02 17:11:44 +02:00
Rafał Miłecki
966ba6daa4 kernel: fix downloading rcX releases
They are no longer stored in the "testing" subdirectory and are not
available as .tar.xz archives. If -rc is detected download it from the
git.kernel.org and use .tar.gz.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2018-12-03 09:34:57 +01:00
Koen Vandeputte
d29e47f191 config: don't define the same symbol twice
In commit fce35bce0f ("config: support new symbol intro'd in kernel
4.12")
I forgot to remove the initial debug test line.

This clearly is wrong as the same symbol is defined conditionally in the
line below as it should be.

I looked over it as I just checked if the symbol was present now upon
testing it.

Fixes: fce35bce0f ("config: support new symbol intro'd in kernel
4.12")

Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
2018-01-17 11:07:17 +01:00
Koen Vandeputte
fce35bce0f config: support new symbol intro'd in kernel 4.12
Symbol CONFIG_INITRAMFS_FORCE allows to ignore the value passed by the
bootloader.

By default, all symbols containing INITRAMFS are wiped from the final
config and then re-added conditionally.

Add support for this symbol, as the build will stop otherwise
questioning the user about this option:

* Restart config...
*
*
* General setup
*
Cross-compiler tool prefix (CROSS_COMPILE) []
Compile also drivers which will not load (COMPILE_TEST) [N/y/?] n

...

Initial RAM filesystem and RAM disk (initramfs/initrd) support
(BLK_DEV_INITRD) [Y/n/?] y
Initramfs source file(s) (INITRAMFS_SOURCE) []
Ignore the initramfs passed by the bootloader (INITRAMFS_FORCE)
[N/y/?] (NEW)

Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
2018-01-13 16:27:45 +01:00
Alexander Couzens
e5fc15bf9a build: move definition of KBUILD_BUILD_TIMESTAMP to include/kernel.mk
Fixes: 0aed054bec (build: add KERNEL_MAKE and KERNEL_MAKE_FLAGS
variables and move to kernel.mk)

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
2017-06-17 15:09:57 +02:00
Felix Fietkau
0aed054bec build: add KERNEL_MAKE and KERNEL_MAKE_FLAGS variables and move to kernel.mk
This allows packages to use kernel make options without the forced
-C $(LINUX_DIR). It also makes it more clear that it to be called from
kernel module packages directly.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-07 18:31:10 +02:00
Florian Fainelli
b6746a6ffb include: Do not alter KERNELRELEASE for external/git kernels
In case we use external and/or git cloned kernels, let the kernel
determine the appropriate KERNELRELEASE. We cannot used
LINUX_UNAME_VERSION because that one gets determined at a later time,
when the kernel is already built proper.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-05-25 09:22:43 +02:00
Florian Fainelli
d0a6340717 Revert "kernel: prevent addition of scm marker to localversion"
This reverts commit 0df2c6563a since it
gets in the way of identifying properly which kernel we are running.
This is particularly important if LEDE is using external kernels/git
cloned kernels. We want to make sure we only load modules from that
specific kernel.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
2017-05-25 09:22:43 +02:00
Rafał Miłecki
f6433eede7 kernel: move initramfs's init script out of base-files
Keeping it in base-files was resulting in adding it to the base-files
package. This file is meant to be included manually for initramfs
images only.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2017-04-04 14:06:40 +02:00
Felix Fietkau
370d740647 kernel: do not try to copy vmlinux out of arch/$(ARCH)/boot
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-03-22 11:43:22 +01:00
Felix Fietkau
1f12a3daaa kernel: speed up build system by getting rid of redundant work
KERNELRELEASE contains a $(shell) call which is evaluated over and over
again.
The call to checksyscalls.sh is unnecessary for LEDE and also takes a
few seconds to complete.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-03-16 19:14:09 +01:00
Felix Fietkau
84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Florian Fainelli
9a08c0ba80 include/kernel: Switch to git download method
Utilize the existing git download logic from include/download.mk and migrate
the kernel download over to it. This avoids repeatedly cloning kernel sources
after a make target/linux/clean for instance.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [fix build error]
2016-12-04 11:41:51 +01:00
Felix Fietkau
685ed1e072 kernel: add STAGING_DIR_HOST/lib to host library search path
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-09-29 13:35:11 +02:00
Felix Fietkau
f5860f898a build: use perl instead of GNU date for KBUILD_BUILD_TIMESTAMP
Fixes warning on non-Linux systems

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-23 11:56:13 +02:00
Josua Mayer
57ef81d78f kernel: prevent adding custom string to localversion
When the kernel build picks up a localversion file in the source tree,
that string is unconditionally appended to LOCALVERSION and affects the
uname string.
Make sure to delete any such file.

Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
2016-08-18 09:49:18 +02:00
Josua Mayer
0df2c6563a kernel: prevent addition of scm marker to localversion
When building the kernel from a git repository, the kernel build appends
either a + or a short commit hash to localversion.
This behaviour can be prevented by passing the empty LOCALVERSION variable
to make.

Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
2016-08-18 09:49:18 +02:00
Jonas Gorski
86ec410418 kernel: check SOURCE_DATE_EPOCH before setting KBUILD_BUILD_TIMESTAMP
Make sure SOURCE_DATE_EPOCH actually contains something.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-08-13 15:49:26 +02:00
Jonas Gorski
5fe923b15d kernel: allow reproducable builds
Similar how we fix the file times in the filesystems, fix the build time
of the kernel, and make the build number static. This should allow the
kernel build to be reproducable when combined with setting the
KERNEL_BUILD_USER and _DOMAIN in case of different machines.

The reproducability only applies to non-initramfs kernels, those still
require additional changes.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
2016-08-13 11:35:00 +02:00
Felix Fietkau
22ef1c83b3 kernel: make the kernel build auto-clean the build dir like package builds
Previous behavior can be restored by using QUILT=1 on target/prepare

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-05 14:17:08 +02:00
Felix Fietkau
9ae952cf8c build: split scripts/metadata.pl into target-metadata.pl and package-metadata.pl
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-06-07 08:58:40 +02:00
Florian Fainelli
09344fde2a kernel: Revert "kernel: set root on NFS when enabled"
This reverts r48591, users needing NFS root will know how to configure
their platform and kernel command-line appropriately.

Reported-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Florian Fainelli <florian@openwrt.org>

SVN-Revision: 48689
2016-02-11 03:05:29 +00:00
Felix Fietkau
108a499c81 kernel: enable CONFIG_KALLSYMS_UNCOMPRESSED=y
It was enabled ages ago when it was added.
It still saves about 10k after LZMA, so let's enable it again.

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48680
2016-02-08 18:44:21 +00:00
Florian Fainelli
d9fab87607 kernel: set root on NFS when enabled
Replace the configured root device with a NFS root device and automatic
IP configuration for booting over NFS.

Signed-off-by: Florian Fainelli <florian@openwrt.org>

SVN-Revision: 48591
2016-02-01 01:06:56 +00:00
Felix Fietkau
b3f7902a06 include/kernel: add custom USER/DOMAIN config options
These allow the generated kernel's build metadata to be defined explicitly.
This metadata is reported, eg, at boot time and in `uname -a` on running
systems. If the variables aren't configured, the current build system username
and hostname are used as normal.

The motivation for this option is to achive reproducible (bit-for-bit
identical) kernel builds of official openwrt releases.

Signed-off-by: bryan newbold <bnewbold@robocracy.org>
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48541
2016-01-28 22:42:34 +00:00
Felix Fietkau
4ab26a52ef build: skip rebuild of kernel images if the source did not change
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48145
2016-01-06 18:38:49 +00:00