Commit Graph

78 Commits

Author SHA1 Message Date
Roman Yeryomin
2ca084ccaa build: improve ccache support
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.

Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2020-07-11 15:19:53 +02:00
Jo-Philipp Wich
efaaadb49e sdk: use bundle-libraries.sh to ship kernel objtool tools
Ensure that the kernel objtool utilities are processed by the library
bundler in order to ensure that they're usable on foreign systems with
different libc versions.

Fixes: a9f6fceb42 ("sdk: fix building external modules when CONFIG_STACK_VALIDATION=y")
Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2019-09-03 10:45:25 +02:00
Petr Štetiar
200f2666fb sdk: fix GCC and Python dangling symlinks
Force prereq again in SDK in order to fix GCC and Python dangling
symlinks:

 staging_dir/host/bin/g++ -> /builder/ath79_generic/ccache_cxx.sh
 staging_dir/host/bin/gcc -> /builder/ath79_generic/ccache_cc.sh
 staging_dir/host/bin/python -> /usr/bin/python3.5
 staging_dir/host/bin/python3 -> /usr/bin/python3.5

Ref: FS#2424
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-08-12 10:04:50 +02:00
Jeffery To
6e300f6a0c sdk: Fix cryptodev-linux build when CONFIG_ARM64_MODULE_PLTS=y
When CONFIG_ARM64_MODULE_PLTS=y, arch/arm64/kernel/module.lds is
required to build cryptodev-linux. This updates the sdk to include this
file.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-08-04 22:09:20 +02:00
Yousong Zhou
a9f6fceb42 sdk: fix building external modules when CONFIG_STACK_VALIDATION=y
The option is at the moment enabled for x86/64.  The selection chain
should be

	config RETPOLINE
	  select STACK_VALIDATION if HAVE_STACK_VALIDATION
	config x86
	  select HAVE_STACK_VALIDATION            if X86_64

cryptdev-linux:

	make[4]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10'
	make -C /store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.113 M=/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10 ARCH=x86 CROSS_COMPILE=x86_64-openwrt-linux-musl- modules
	make[5]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.113'
	make[6]: *** No rule to make target '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10/ioctl.o', needed by '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10/cryptodev.o'.  Stop.
	Makefile:1533: recipe for target '_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/cryptodev-linux-cryptodev-linux-1.10' failed

Open vSwitch intree kmods

	make[7]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux'
	make -C /store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111 ARCH=x86 M=/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux modules
	make[8]: Entering directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111'
	make[9]: *** No rule to make target 'tools/objtool/objtool', needed by '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux/actions.o'.  Stop.
	Makefile:1533: recipe for target '_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux' failed
	make[8]: *** [_module_/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/openvswitch-2.11.0/datapath/linux] Error 2
	make[8]: Leaving directory '/store/buildbot/slave/x86_64/build/sdk/build_dir/target-x86_64_musl/linux-x86_64/linux-4.14.111'

Reference: builddeb: include objtool binary in headers package, https://patchwork.kernel.org/patch/9014071/
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-04-26 13:05:39 +00:00
Karl Vogel
ae980458ab sdk: find kernel modules when KDIR is a symlink
The find statement would not return any results if the KDIR_BASE pointed to a
symlink. Ran into this issue due to a custom Kernel/Prepare that was installing
a symlink to the kernel directory.

The extra slash at the end fixes this scenario and does no harm for targets that
have a proper KDIR.

Signed-off-by: Karl Vogel <karl.vogel@gmail.com>
2018-11-29 09:36:14 +01:00
Jo-Philipp Wich
4bb8a678e0 sdk: include arch/arm/ Linux includes along with arch/arm64/ ones
The Linux headers on arm64 architectures contain references to common
arch/arm/ headers which were not bundled by the SDK so far.

Check if we're packing the SDK for an arm64 target and if we do, also
include arch/arm headers as well.

Fixes FS#1725.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-08-03 13:47:29 +02:00
Jo-Philipp Wich
d0e0b7049f sdk: bundle usbip userspace sources
Bundle the usbip utility sources shipped with the Linux kernel tree in
order to allow the usbip packages from the package feed to build within
the OpenWrt SDK.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-08-01 09:12:17 +02:00
Jo-Philipp Wich
17b194aeac sdk: change base feed fallback to git.openwrt.org
Change the hardcoded lede-project.org base feed fallback to
$(PROJECT_GIT)/openwrt/openwrt.git instead.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-11 18:20:07 +01:00
Jo-Philipp Wich
f839ad685b sdk: switch base feed url to git.openwrt.org
As LEDE is rebranding to OpenWrt now, adjust the base feed Git reference
accordingly.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-04 13:44:42 +01:00
Rafał Miłecki
deaf9597c6 sdk: restrict base feed repo to public git for CONFIG_BUILDBOT only
This allows people to build SDK from custom repository (git access using
ssh) and keep original URL in SDK's feeds.conf.default.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2017-12-05 22:56:31 +01:00
Felix Fietkau
9467ce42da build: get rid of host.mk
Defined required host related variables in toplevel.mk instead

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-26 13:31:44 +01:00
Jo-Philipp Wich
7c9f5c7e6b sdk: emit proper tag references for base URLs
Properly resolve symbolic tag names when constructing the base feed Git url
and avoid emitting "HEAD" references when building from detached commits.

Fixes #495, #501.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-02-19 20:47:49 +01:00
Felix Fietkau
90728c796d sdk: clean scripts/config before packing tarball (FS#504)
Avoids shipping potentially incompatible object files and also reduces
the tarball size

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-13 11:46:14 +01:00
Felix Fietkau
412cf4cc51 sdk: add missing change from r3328-f55c29e4c7
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-08 12:01:59 +01:00
Jo-Philipp Wich
beb69e242e sdk: explicitely remove ccache directories when packing SDK
Upon first invocation, the ccache program will create the required directory
hierarchy so there is no point in shipping these empty directories.

Removing those paths also avoids shipping dangling symlinks in case the
directories got linked elsewhere, e.g. into a shared global cache.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-24 15:31:12 +01:00
Jo-Philipp Wich
dab6338b47 sdk: fix Git URL detection
Instead of relying on complex sed patterns that trip up make syntax rules, use
GNU Makes builtin filter function to match desired URLs.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-20 10:10:54 +01:00
Jo-Philipp Wich
ada6d9feca sdk: avoid using private repository clone urls as base repo entry
Only consider the repository origin url as valid base feed entry if it is a
git://, http:// or https:// location.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-19 20:46:58 +01:00
Jo-Philipp Wich
a68b1b9c8a sdk: do not strip static libraries
Do not strip static libraries shipped with the SDK in order to preserve the
archive index. If we strip the index of the shipped libraries, host programs
will fail to link these libraries with errors like:

    libssl.a: error adding symbols: Archive has no index; run ranlib to add one

The error was found while investigating a Python host build failure within
the SDK environment.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-18 03:18:53 +01:00
Felix Fietkau
3e521fa112 sdk: exclude locale files to save some space
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-13 19:46:45 +01:00
Jo-Philipp Wich
72d751cba9 build: rework library bundling
Rework the bundle-libraries.sh implementation to use a more robust approach
for executing host binaries through the shipped ELF loader and libraries.

The previous approach relied on symlinks pointing to a wrapper script which
caused various issues, especially with multicall binaries as the original
argv[0] name was not preserved through the ld.so invocation. Another down-
side was the fact that the actual binaries got moved into another directory
which caused executables to fail looking up resources with paths relative
to the executable location.

The new library wrapper implements the following improvements:

 - Instead of symlinks pointing to a common wrapper, each ELF executable
   is now replaced by a unqiue shell script which retains the original
   program name getting called

 - Instead of letting ld.so invoke the ELF executable directly, launch
   the final ELF binary through a helper program which fixes up the argv[0]
   argument for the target program

 - Support sharing a common location for the bundled libraries instead of
   having one copy in each directory containing wrapped binaries

Finally modify the SDK build to wrap the staging_dir and toolchain binaries
which allows to use the SDK on systems with a different glibc version.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-01-10 12:27:28 +01:00
Mathias Kresin
59261cbf38 docs: remove all refrences in Makefiles/scripts
The SDK Makefile still trys to copy the docs folder which was removed
with 882f4d2d63. This causes an SDK build
error.

All other removals are just cleanup.

Signed-off-by: Mathias Kresin <dev@kresin.me>
2016-11-30 10:13:14 +01:00
Daniel Engberg
9f61ccd9e3 target/sdk: Switch to xz compression instead of bz2
Switch to xz compression instead of using bz2.
This makes a considerable difference in size, ar71xx SDK from 60M to 35M.

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
2016-11-04 11:38:19 +01:00
Jo-Philipp Wich
8cc9224115 sdk: predefine SOURCE_DATE_EPOCH
When building packages within the SDK, there is no Git revision history
available so prepopulate SOURCE_DATE_EPOCH in version.mk, similar to
how we handle REVISION already.

Acked-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-10-21 12:43:45 +02:00
Daniel Dickinson
71370d2c55 target/{sdk,imagebuild}: Fix for symlink-tree
With symlink tree some directories are just symlinked which
means IB and SDK end up with a symlink instead of an actual
directory; this fixes the missing files by dereferencesing
the directories instead of copying the symlinks.

Signed-off-by: Daniel Dickinson <lede@cshore.thecshore.com>
2016-09-30 00:39:25 +02:00
Felix Fietkau
847cb10f47 target/sdk: ship toolchain and kernel module package
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-01 12:00:25 +02:00
Alexander Couzens
2e980479c1 IB/SDK/toolchain: use lower cases filenames
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
2016-06-01 17:54:36 +02:00
Alexander Couzens
4a7c653400 IB/SDK/toolchain: use VERSION_DIST_SANITIZED instead of VERSION_DIST
VERSION_DIST can contains spaces which produces problems when used as file name

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
2016-06-01 17:54:36 +02:00
Jo-Philipp Wich
9eeb267c06 sdk: stop shipping a .config
Now that the most important options are exposed via menuconfig in the SDK we
can stop shipping a (possibly outdated) copy of .config from the emitting
buildroot.

Leaving out .config has two positive side effects:

 - A simple "make" run without any prior config will launch menuconfig, forcing
   interactive users to make appropriate choices

 - Buildbot systems (e.g. our phase2 builder) can issue a "make defconfig" to
   obtain a .config enabling all packages by default

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-05-24 23:40:34 +02:00
Jo-Philipp Wich
484cb91ad5 sdk: bundle required firmware files
Bundle all firmware files which have been built along with the kernel to fix
packaging of kmods in the SDK environment that rely on in-tree firmwares,
specifically kmod-e100.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-21 16:34:39 +02:00
Jo-Philipp Wich
f8ce7e028d sdk: do not exclude ccache executable
Change tar exclude pattern to skip ccache directory contents but not the
ccache executable itself.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-18 12:28:16 +02:00
Jo-Philipp Wich
0f6f518e7a sdk: fix generation of base feed url
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-17 21:50:31 +02:00
Jo-Philipp Wich
2224dbdb9d sdk: add base repository fallback
When the build system cannot infer its own repository url then let it fall
back to http://git.openwrt.org/ for the base feed.

The path to openwrt.git is guessed from the VERSION_NUMBER variable:

 "unknown" or "r12345" -> http://git.openwrt.org/openwrt.git
 "15.05.1"             -> http://git.openwrt.org/15.05/openwrt.git

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 48212
2016-01-12 09:25:45 +00:00
Felix Fietkau
6318ff744d target/sdk: include modules.builtin, it is necessary for packaging kernel modules
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48206
2016-01-11 23:11:09 +00:00
Felix Fietkau
7dfbac8073 target/sdk: do not include the target staging dir anymore, it is unnecessary now
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48157
2016-01-08 23:18:18 +00:00
Felix Fietkau
4a7340d8c6 target/sdk: ship .ko files to allow rebuild of the base kmod-* packages
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48156
2016-01-08 23:18:14 +00:00
Felix Fietkau
def6be413c target/sdk: Fix filename with external toolchain
When building SDK with external toolchain, don't
include _gcc in the filename

Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>

SVN-Revision: 48084
2016-01-03 14:45:18 +00:00
Felix Fietkau
d69efa4e15 targets: Use configured distribution name for SDK, IB, and Toolchain tarball names
We allow to configure the version distribution name;
let's also use it for the tarballs (SDK, ImageBuilder,
and SDK).

Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>

SVN-Revision: 48081
2016-01-03 14:44:42 +00:00
Jo-Philipp Wich
49fda5e4d9 sdk: use prepare target to initialize git snapshot
The new approach is less prone to race conditions and will properly
snapshot the symlinks prepared by the prereq-build target.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 44330
2015-02-08 22:54:27 +00:00
Jo-Philipp Wich
f52a62d600 sdk: rename file to mention target/subtarget and move host system to suffix
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 44312
2015-02-07 21:01:43 +00:00
Jo-Philipp Wich
15d8db1f8c build: add version number to filenames
This commit introduces a new option CONFIG_VERSION_FILENAMES which causes
OpenWrt to embed the version number in generated image files, SDK- and
ImageBuilder archives.

The option is enabled by default if CONFIG_VERSIONOPT is set.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 43869
2015-01-08 11:02:00 +00:00
Felix Fietkau
db9f9a6407 sdk: add missing kernel include files
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43621
2014-12-11 17:03:04 +00:00
Jo-Philipp Wich
cc97eaff10 sdk: improve feed setup
- Add current buildroot scm url as base feed to feeds.conf
 - Prefer feeds.conf over feeds.conf.default when generating the SDK feeds.conf.default

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42935
2014-10-16 16:28:12 +00:00
Jo-Philipp Wich
057e518224 sdk: predefine kernel version magic
Kernel modules which are built using the SDK do not have access
to the .vermagic file in the Linux build dir, therefore the
generated .ipk metadata depends on kernel versions like
'3.10.49-1-unknown' which are not satisfiable by the kernel
meta package in the image.

Fix this problem by substituting 'unknown' with the current
version magic in include/kernel.mk when packing the SDK.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42590
2014-09-17 10:25:47 +00:00
Felix Fietkau
734f7609b9 sdk: keep $(STAGING_DIR)/usr/bin, at least curl needs it
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 42397
2014-09-01 22:34:37 +00:00
Felix Fietkau
2a5f6cadd1 target/sdk: strip host binaries before packing
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39915
2014-03-13 14:14:00 +00:00
Felix Fietkau
8c9fed34e4 target/sdk: leave out some unnecessary directories
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39914
2014-03-13 14:13:57 +00:00
Felix Fietkau
cedfe135ab target/sdk: generate a Config.in file with the settings of the build that the SDK was generated from
This allows make oldconfig/menuconfig to run

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39864
2014-03-10 18:58:49 +00:00
Felix Fietkau
23df56c490 target/sdk: remove obsolete compatibility makefiles
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39863
2014-03-10 18:58:44 +00:00
Felix Fietkau
5ef90f6540 target/sdk: remove obsolete exclude line
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39861
2014-03-10 18:58:34 +00:00