Commit Graph

120 Commits

Author SHA1 Message Date
Chris Osgood
c5e167e0d6
build: fix warnings from grep
Fixes build warnings when using newer versions of grep.

Signed-off-by: Chris Osgood <chris_github@functionalfuture.com>
Tested-by: Georgi Valkov <gvalkov@gmail.com>
2022-09-28 17:17:48 +02:00
Hauke Mehrtens
9403810c02 toolchain: Select USE_SSTRIP with external musl toolchain
When we use the internal toolchain USE_SSTRIP will be selected by
default for musl libc and USE_STRIP when glibc is used. Do the same when
an external toolchain is used. USE_GLIBC will also be set for external
toolchain builds based on the EXTERNAL_TOOLCHAIN_LIBC_USE_GLIBC setting.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-09-12 23:16:22 +02:00
Rosen Penev
22b7bd6b13
Makefile: replace head call with grep's -m
head is not necessary here.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-09-10 15:42:36 +02:00
Christian Lamparter
b479db9062 sdk: add spidev-test to the bundle of userspace sources
moves and extends the current facilities, which have been
added some time ago for the the usbip utility, to support
more utilites that are shipped with the Linux kernel tree
to the SDK.

this allows to drop all the hand-waving and code for
failed previous attempts to mitigate the SDK build failures.

Fixes: bdaaf66e28 ("utils/spidev_test: build package directly from Linux")
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2022-07-17 14:19:21 +02:00
Eneas U de Queiroz
009293c52e build: scripts/config - update to kconfig-v5.14
Functional Changes
---------- -------
- make 'imply' not impose any restrictions: allow symbols implied by y
  to become m
- change "modules" from sub-option to first-level attribute

Bugfixes
--------
- nconf: fix core dump when searching in empty menu
- nconf: stop endless search loops
- xconfig: fix content of the main widget
- xconfig: fix support for the split view mode

Other Changes
----- -------
- highlight xconfig 'comment' lines with '***'
- xconfig: navigate menus on hyperlinks
- xconfig: drop support for Qt4
- improve host ncurses detection

Update the 'option modules' usage to just 'modules' in Config.in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-19 13:10:01 +01:00
Felix Fietkau
0ac0840088 sdk: ship llvm toolchain
This allows ebpf packages like qosify to be built with it

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-01-27 11:32:56 +01:00
Ansuel Smith
88204bfa82 treewide: drop use of which
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-01-17 09:14:26 +01:00
Felix Fietkau
2a832b3df5 llvm-bpf: support creating a tarball in bin
This can be used for adding the toolchain to an existing tree without having
to build it from scratch.
Enable building the toolchain + tarball by default on buildbot

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-04 16:59:22 +01:00
Sven Roederer
1e4b191ac8 sdk: unset BINARY_FOLDER and DOWNLOAD_FOLDER in final archives
Using these config-options to customize the folders used at build-time makes these
folder settings appear in generated archive. This causes the SDK to be not
portable, as it's going to use the build-time folders on the new systems.
The errors vary from passing the build, disk out-of-space to permission denied.

The build-time settings of these folders are passed into the archive via Config.build.
The expected behavior is that the SDK acts after unpacking like these settings have
their defaults, using intree folders. So just filter these folders out when running
convert-config.pl to create Config.build.

This addresses the same issue that's fixed in the previous commit for the imagebuilder.

Signed-off-by: Sven Roederer <devel-sven@geroedel.de>
2021-05-08 12:14:04 +02:00
David Bauer
bb817bb4b8 sdk: expose binary strip settings
Expose the SDK options for binary stripping to the menuconfig. This
way, packages can easily be built with debug symbols using the SDK.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-02-20 01:29:36 +01:00
Adrian Schmutzler
4894a3ff28 target: fix copying of licenses for SDK/toolchain
The SDK and target/toolchain copy the license file into their
directories. During the rename/move from LICENSE to a LICENSES
folder, this has not been updated.

Update it now, and include the new COPYING file.

While at it, improve formatting/indent.

Fixes: 882e301461 ("LICENSES: include all used licenses in
LICENSES directory")

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-14 21:56:29 +01:00
Jo-Philipp Wich
7e4585e593 sdk: invoke bundle-libraries.sh w/o buildroot dirs in $PATH
Invoke bundle-libraries.sh with any buildroot related directory entries
removed from $PATH to avoid picking up cross versions of utilities like
ldd which will not properly work when used against host executables.

This should fix executable bundling for glibc-target SDKs.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-12-29 01:07:42 +01:00
Petr Štetiar
ec0ec0428e sdk: fix missing include directories
It's not possible to compile some applications which are using
`-Werror=missing-include-dirs` compiler flags with the SDK as some
target directories are missing in the SDK tarball:

 cc1: error: staging_dir/target/usr/include: No such file or directory [-Werror=missing-include-dirs]
 cc1: error: staging_dir/target/include: No such file or directory [-Werror=missing-include-dirs]

Fix this by adding the missing directories in the SDK.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-11-02 08:39:49 +01:00
Felix Fietkau
a4c1c169c3 sdk: keep the linux package in the same subdir as in the main build system
Fixes: 953435795d ("build: always build package/kernel/linux")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2020-10-26 02:36:02 +01:00
Paul Spooren
8c9a788172
build: SDK/IB reproducible and faster compression
Both IB and SDK now use the same logic for packing.

This commit add reproducible multithread compression to the SDK and
corrects the file mtime for both. Previously all files where just copied
over from the build system, generating random mtimes.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-23 18:55:49 +02:00
Roman Yeryomin
2ca084ccaa build: improve ccache support
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.

Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2020-07-11 15:19:53 +02:00
Eneas U de Queiroz
94e27d62ab sdk: add OpenWrt branding to menuconfig & .config
Set the mainmenu symbol in SDK Config.in to "OpenWrt Configuration", the
same as the main OpenWrt Config.in.  This string is is used as the name
of the top menu in menuconfig, and at the top of the .config file.  If
unset, current kconfig will use "Linux Kernel Configuration".

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-05-20 17:03:45 +02:00
Eneas U de Queiroz
1f2539a1f4 sdk: fix host menu config targets using ncurses
This applies 965f341aa9 ("build: fix host menu config targets using
ncurses") to the SDK top Makefile.

If there is a pkg-config in the staging dir, it will try to use it
instead of the host system's pkg-config; then it will fail to find the
ncurses package.  Linux's default will be used, which fails in some
cases, such as recent Gentoo systems.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
[fixed From: to match SoB]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-05-20 17:03:45 +02:00
Jo-Philipp Wich
efaaadb49e sdk: use bundle-libraries.sh to ship kernel objtool tools
Ensure that the kernel objtool utilities are processed by the library
bundler in order to ensure that they're usable on foreign systems with
different libc versions.

Fixes: a9f6fceb42 ("sdk: fix building external modules when CONFIG_STACK_VALIDATION=y")
Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-09-03 10:45:25 +02:00
Petr Štetiar
200f2666fb sdk: fix GCC and Python dangling symlinks
Force prereq again in SDK in order to fix GCC and Python dangling
symlinks:

 staging_dir/host/bin/g++ -> /builder/ath79_generic/ccache_cxx.sh
 staging_dir/host/bin/gcc -> /builder/ath79_generic/ccache_cc.sh
 staging_dir/host/bin/python -> /usr/bin/python3.5
 staging_dir/host/bin/python3 -> /usr/bin/python3.5

Ref: FS#2424
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-08-12 10:04:50 +02:00
Jeffery To
6e300f6a0c sdk: Fix cryptodev-linux build when CONFIG_ARM64_MODULE_PLTS=y
When CONFIG_ARM64_MODULE_PLTS=y, arch/arm64/kernel/module.lds is
required to build cryptodev-linux. This updates the sdk to include this
file.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-08-04 22:09:20 +02:00
Yousong Zhou
a9f6fceb42 sdk: fix building external modules when CONFIG_STACK_VALIDATION=y
The option is at the moment enabled for x86/64.  The selection chain
should be

	config RETPOLINE
	  select STACK_VALIDATION if HAVE_STACK_VALIDATION
	config x86
	  select HAVE_STACK_VALIDATION            if X86_64

cryptdev-linux:

	make[4]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10'
	make -C /store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.113 M=/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10 ARCH=x86 CROSS_COMPILE=x86_64-openwrt-linux-musl- modules
	make[5]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.113'
	make[6]: *** No rule to make target '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10/ioctl.o', needed by '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10/cryptodev.o'.  Stop.
	Makefile:1533: recipe for target '_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10' failed

Open vSwitch intree kmods

	make[7]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux'
	make -C /store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111 ARCH=x86 M=/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux modules
	make[8]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111'
	make[9]: *** No rule to make target 'tools/objtool/objtool', needed by '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux/actions.o'.  Stop.
	Makefile:1533: recipe for target '_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux' failed
	make[8]: *** [_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux] Error 2
	make[8]: Leaving directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111'

Reference: builddeb: include objtool binary in headers package, https://patchwork.kernel.org/patch/9014071/
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-04-26 13:05:39 +00:00
Karl Vogel
ae980458ab sdk: find kernel modules when KDIR is a symlink
The find statement would not return any results if the KDIR_BASE pointed to a
symlink. Ran into this issue due to a custom Kernel/Prepare that was installing
a symlink to the kernel directory.

The extra slash at the end fixes this scenario and does no harm for targets that
have a proper KDIR.

Signed-off-by: Karl Vogel <karl.vogel@gmail.com>
2018-11-29 09:36:14 +01:00
Tomasz Maciej Nowak
d93b09fa74 sdk: expose part of advanced configuration options
These option can be useful in developement/packagig applications for
OpenWrt. Instead of searching them by hand in Config-build.in lets
expose them for easier access.

Signed-off-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
2018-11-26 12:07:06 +01:00
Jo-Philipp Wich
4bb8a678e0 sdk: include arch/arm/ Linux includes along with arch/arm64/ ones
The Linux headers on arm64 architectures contain references to common
arch/arm/ headers which were not bundled by the SDK so far.

Check if we're packing the SDK for an arm64 target and if we do, also
include arch/arm headers as well.

Fixes FS#1725.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-08-03 13:47:29 +02:00
Jo-Philipp Wich
d0e0b7049f sdk: bundle usbip userspace sources
Bundle the usbip utility sources shipped with the Linux kernel tree in
order to allow the usbip packages from the package feed to build within
the OpenWrt SDK.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-08-01 09:12:17 +02:00
Jo-Philipp Wich
17b194aeac sdk: change base feed fallback to git.openwrt.org
Change the hardcoded lede-project.org base feed fallback to
$(PROJECT_GIT)/openwrt/openwrt.git instead.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-11 18:20:07 +01:00
Jo-Philipp Wich
f839ad685b sdk: switch base feed url to git.openwrt.org
As LEDE is rebranding to OpenWrt now, adjust the base feed Git reference
accordingly.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-04 13:44:42 +01:00
Zoltan HERPAI
7b5c989ab9 merge: targets: update image generation and targets
Signed-off-by: Zoltan HERPAI <wigyori@uid0.hu>
2017-12-08 19:41:18 +01:00
Rafał Miłecki
deaf9597c6 sdk: restrict base feed repo to public git for CONFIG_BUILDBOT only
This allows people to build SDK from custom repository (git access using
ssh) and keep original URL in SDK's feeds.conf.default.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2017-12-05 22:56:31 +01:00
Felix Fietkau
9467ce42da build: get rid of host.mk
Defined required host related variables in toplevel.mk instead

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Jo-Philipp Wich
7c9f5c7e6b sdk: emit proper tag references for base URLs
Properly resolve symbolic tag names when constructing the base feed Git url
and avoid emitting "HEAD" references when building from detached commits.

Fixes #495, #501.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-02-19 20:47:49 +01:00
Felix Fietkau
9dcb921d90 build: add buildbot specific config option for setting defaults
This can be used to tweak the buildbot behavior without having to change
buildbot's configuration.
It will also allow us to add more aggressive clean steps (e.g. on
toolchain changes), which would break developers' workflows if enable
by default.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-14 14:18:35 +01:00
Felix Fietkau
90728c796d sdk: clean scripts/config before packing tarball (FS#504)
Avoids shipping potentially incompatible object files and also reduces
the tarball size

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-13 11:46:14 +01:00
Felix Fietkau
412cf4cc51 sdk: add missing change from r3328-f55c29e4c7
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-08 12:01:59 +01:00
Jo-Philipp Wich
beb69e242e sdk: explicitely remove ccache directories when packing SDK
Upon first invocation, the ccache program will create the required directory
hierarchy so there is no point in shipping these empty directories.

Removing those paths also avoids shipping dangling symlinks in case the
directories got linked elsewhere, e.g. into a shared global cache.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-24 15:31:12 +01:00
Jo-Philipp Wich
dab6338b47 sdk: fix Git URL detection
Instead of relying on complex sed patterns that trip up make syntax rules, use
GNU Makes builtin filter function to match desired URLs.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-20 10:10:54 +01:00
Jo-Philipp Wich
ada6d9feca sdk: avoid using private repository clone urls as base repo entry
Only consider the repository origin url as valid base feed entry if it is a
git://, http:// or https:// location.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-19 20:46:58 +01:00
Jo-Philipp Wich
a68b1b9c8a sdk: do not strip static libraries
Do not strip static libraries shipped with the SDK in order to preserve the
archive index. If we strip the index of the shipped libraries, host programs
will fail to link these libraries with errors like:

    libssl.a: error adding symbols: Archive has no index; run ranlib to add one

The error was found while investigating a Python host build failure within
the SDK environment.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-18 03:18:53 +01:00
Felix Fietkau
3e521fa112 sdk: exclude locale files to save some space
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-13 19:46:45 +01:00
Jo-Philipp Wich
72d751cba9 build: rework library bundling
Rework the bundle-libraries.sh implementation to use a more robust approach
for executing host binaries through the shipped ELF loader and libraries.

The previous approach relied on symlinks pointing to a wrapper script which
caused various issues, especially with multicall binaries as the original
argv[0] name was not preserved through the ld.so invocation. Another down-
side was the fact that the actual binaries got moved into another directory
which caused executables to fail looking up resources with paths relative
to the executable location.

The new library wrapper implements the following improvements:

 - Instead of symlinks pointing to a common wrapper, each ELF executable
   is now replaced by a unqiue shell script which retains the original
   program name getting called

 - Instead of letting ld.so invoke the ELF executable directly, launch
   the final ELF binary through a helper program which fixes up the argv[0]
   argument for the target program

 - Support sharing a common location for the bundled libraries instead of
   having one copy in each directory containing wrapped binaries

Finally modify the SDK build to wrap the staging_dir and toolchain binaries
which allows to use the SDK on systems with a different glibc version.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-10 12:27:28 +01:00
Mathias Kresin
59261cbf38 docs: remove all refrences in Makefiles/scripts
The SDK Makefile still trys to copy the docs folder which was removed
with 882f4d2d63. This causes an SDK build
error.

All other removals are just cleanup.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-11-30 10:13:14 +01:00
Daniel Engberg
9f61ccd9e3 target/sdk: Switch to xz compression instead of bz2
Switch to xz compression instead of using bz2.
This makes a considerable difference in size, ar71xx SDK from 60M to 35M.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
2016-11-04 11:38:19 +01:00
Jo-Philipp Wich
8cc9224115 sdk: predefine SOURCE_DATE_EPOCH
When building packages within the SDK, there is no Git revision history
available so prepopulate SOURCE_DATE_EPOCH in version.mk, similar to
how we handle REVISION already.

Acked-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-10-21 12:43:45 +02:00
Daniel Dickinson
71370d2c55 target/{sdk,imagebuild}: Fix for symlink-tree
With symlink tree some directories are just symlinked which
means IB and SDK end up with a symlink instead of an actual
directory; this fixes the missing files by dereferencesing
the directories instead of copying the symlinks.

Signed-off-by: Daniel Dickinson <lede@cshore.thecshore.com>
2016-09-30 00:39:25 +02:00
Felix Fietkau
d4abe72cce target/sdk: update README.SDK to explain dependency handling
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-01 12:00:27 +02:00
Felix Fietkau
847cb10f47 target/sdk: ship toolchain and kernel module package
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-01 12:00:25 +02:00
Felix Fietkau
12703d5b29 sdk: provide a config symbol for detecting builds within the SDK
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-31 12:25:25 +02:00
Daniel Dickinson
3c6b091b65 sdk: Allow to configure signed pacakge lists
Add option to enable signing packages lists in SDK.

Signed-off-by: Daniel Dickinson <lede@daniel.thecshore.com>
2016-07-13 20:03:10 +02:00
Daniel Dickinson
9da1bf58b6 sdk: Fix keeping unset as unset
The configuration preservation mechanism fails to keep items was are unset
as unset.  For busybox this causes issues when building with custom
settings in the SDK.  Therefore preserve busybox unset settings.
In addition we preserve non-package selection unset settings because
they may be important for making sure we compile and identical
package for packages that are recompiled in the SDK.

Now that SDK not longer ships .config this becomes relevant for
any use of the SDK.

Signed-off-by: Daniel Dickinson <lede@daniel.thecshore.com>
2016-06-21 13:36:23 +02:00