Commit Graph

1058 Commits

Author SHA1 Message Date
Christian Marangi
26bb4b409d
scripts: ext-tools: add option to only refresh timestamps
It's possible to have prebuilt tools already extracted. Add option to
just refresh the timestamps.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:05 +01:00
Christian Marangi
1506f8c322
scripts: ext-tools: follow links for host tools path
Host tools path may be a symbolic link. Use -H with find to follow path
links passed from command line to find command.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:05 +01:00
Christian Marangi
5f1758ef14
scripts/dl_github_archieve.py: fix generating unreproducible tar
Allign dl_github_archieve.py to 8252511dc0
change. On supported system the sigid bit is applied to files and tar
archieve that on tar creation. This cause unreproducible tar for these
system and these bit should be dropped to produce reproducible tar.

Add the missing option following the command options used in other
scripts.

Fixes: 75ab064d2b ("build: download code from github using archive API")
Suggested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Tested-by: Robert Marko <robimarko@gmail.com>
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-12 14:59:07 +01:00
David Bauer
9ac377d0e0 scripts: add Apache fastly mirror
Add the Apache fastly mirror as preferred download source.
This service is using a dual-stacked CDN.

Signed-off-by: David Bauer <mail@david-bauer.net>
2023-01-07 01:32:58 +01:00
Shiji Yang
9fa8fff0af scripts: remove redundant character '0x0a' from Linksys image signature
The redundant character '0x0a' after the 192 bytes '0x00' padding broke
the factory image. We need to remove it to make things work again.

Fixes: e6769d11f3 scripts: fix missing character '0' issue in linksys image
Tested-by: Tony Butler <spudz76@gmail.com>
Signed-off-by: Shiji Yang <yangshiji66@qq.com>
2022-11-27 13:18:29 +01:00
Andre Heider
9a2d362bf7
scripts: fix dl_cleanup.py argument handling
The -w|--whitelist and -D|--download-dir arguments pass an additional value,
properly evaluate that.
Also allow to pass the download directory without -D|--download-dir, just as
the usage describes.
Finally fix spitting out the wrong error messages about those args.

Signed-off-by: Andre Heider <a.heider@gmail.com>
2022-11-15 14:45:50 +01:00
Felix Fietkau
b8bf27e4cb scripts/mkits.sh: fix portability issue
BSD wc can output more whitespaces, which breaks the cut usage.
Replace the cut invocation with awk, which is more portable.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-11-09 20:43:41 +01:00
David Bauer
a208f0a9be mkits: support definition of DTB loadaddr
Support defining a per-device loadaddress for the DTB. This is required
for devices which to not align the DTB from the bootloader correctly.

Signed-off-by: David Bauer <mail@david-bauer.net>
2022-11-08 12:16:36 +01:00
Shiji Yang
e6769d11f3 scripts: fix missing character '0' issue in linksys image
In the stock firmware of Linksys, there is a '0' after the crc checksum.
Validated on EA6350V3, EA7300 and EA7300V2's stock images.

Fixes: 892d741259 build: add a script for generating Linksys factory images
Signed-off-by: Shiji Yang <yangshiji66@qq.com>
2022-10-30 23:14:45 +01:00
Zhang Hua
a53f29b583
scripts/download.pl: pass aria2 config in ENV only
The aria2c command tries to load config from
${XDG_CONFIG_HOME:-${HOME}/.config}/aria2/aria2.conf by default,
which may result unexpected behavior.

As a replacement, people can use environment variable ARIA2C_OPTIONS
to custom arguments passed to aria2c like curl and wget below.
Including --conf-path=/path/to/config.conf in ARIA2C_OPTIONS can
also set a custom config file path easily if needed.

Signed-off-by: Zhang Hua <zhanghuadedn@gmail.com>
2022-10-20 20:38:31 +02:00
Christian Marangi
f17608ddca
scripts/download.pl: make the download tool configurable
Introduce a new option in the "Advanced configuration options" to
configure a custom download tool.

By declaring a string in "Use custom download tool" an user can force
what command to use to download package. With the string empty the
default tool used is curl, with wget as a fallback if not available.

download.pl supports 3 tools officially aria2c, curl and wget.
If one of the tool is used in this config, download.pl will use the
default args to make use of them.

If the provided string is different than aria2c, curl or wget, the command
is used as is and the download url will be appended at the end of such command.

While at it also tweak the tool selection logic and chose the tool only
once when the script is called and move aria2c specific variables in the
relevant section.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-20 00:35:19 +02:00
Christian Marangi
5428bdc2df
scripts/ext-tools: introduce new script to install prebuilt tools
Add a simple script to make it easier to install a prebuilt tools tar.
Currently it will be used by our tools container and kernel workflow on
github.

Simple script that take a tar that contains prebuilt host tools, extract
them and refresh the timestamps to skip recompilation of such host
tools.

By default it refresh timestamps of build_dir/host and
staging_dir/host/stamp.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-30 22:42:18 +02:00
Bradford Zhang
e937663025
scripts/download.pl: add tsinghua and ustc mirrors
Add https://mirrors.tuna.tsinghua.edu.cn/ and https://mirrors.ustc.edu.cn/ mirrors into download.pl to speed up download in China.

Signed-off-by: Bradford Zhang <zyc@zyc.name>
2022-09-30 14:57:19 +02:00
Christian Marangi
261925a869
scripts/download.pl: generilize and simplify download tool check
Generilize download tool check and skip other check if a download tool
has been found.
While at it also reintroduce c836ca84e8
that was previously dropped with aria2c support.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-28 16:55:10 +02:00
Christian Marangi
f1b3958d02
scripts/download.pl: fix support for aria2c download tool on macos
Currently we use /dev/shm to place aria2c tmp file. This is not present
on macos. Use the openwrt tmp directory instead of the linux-only
/dev/shm to save compatibility with more os.

Fixes: d391236269 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-26 20:50:10 +02:00
Christian Marangi
5df60f5c24
scripts/download.pl: fix mirrors regression for curl and wget
With the introduction of aria2c support, curl and wget no longer try to
download the file from mirrors. Fix this regression by emptying the
remaining mirrors list only when aria2c is used.

Fixes: d391236269 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-26 20:50:09 +02:00
Christian Marangi
295e0ed7a9
scripts/download.pl: fix whitespace in mirror urls and drop for
Fix whitespace in mirror urls and replace for loop with join+map logic.

Fixes: d391236269 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-25 15:54:12 +02:00
Bradford Zhang
d391236269
download.pl: add aria2c support
Use aria2c download tool by default on package download if available in
the system.
aria2c permits to use multiple mirrors and may improve download speed on
special context where servers are hard to reach.

Co-authored-by: Christian Marangi <ansuelsmth@gmail.com>
Signed-off-by: Bradford Zhang <zyc@zyc.name>
[ fix wrong var in the script and improve commit description ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-25 14:56:45 +02:00
Michael Pratt
da4609788d scripts/dl_cleanup: add support for subdirectories
Allow comparing subdirectories exactly like files.

Handle a corner case where the new subdirectory
has the same tarball inside of it
as the one that was downloaded
before a subdirectory for that package was established.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2022-09-22 16:48:36 -04:00
Ansuel Smith
cf2c9498be
scripts: add clean of build dir to dl_cleanup script
Improve dl_cleanup by adding an option to also clean the build directory
related to the downloaded package.
The script will check every directory in build_dir/ and check if any
old package is present there. If outdated package are found, the old
one are cleared leaving only the last one.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:47 +02:00
Ansuel Smith
69760d415d
scripts: assume dl/ the default dir for dl_cleanup script
Assume dl/ the default dl dir and make it configurable if someone have that in
a different place.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:47 +02:00
Ansuel Smith
428c5bf3d1
scripts: add additional regex for dl_cleanup
Add additional regex for dl_cleanup script to handle
case with xxx-v1.2a and xxx-v1.2.3a

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:43 +02:00
Petr Štetiar
50a48faa1b scripts/download.pl: fix downloads with wget
Several users of wget for downloads (curl is not available in the
system) have reported broken download functionality:

 wget --tries=5 --timeout=20 --output-document=-  https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.142.tar.xz
 http://: Invalid host name.

Thats all happening due to '' was passed as an argument, which got later
expanded to http://.

In the context of a list constructor '' is not nothing, it is an empty
string element.  So fix it by using () as it will yield "nothing" and
thus not introduce an empty string element.

Fixes: #10692
Fixes: 90c6e3aedf ("scripts: always check certificates")
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [shellwords() -> ()]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-13 09:14:09 +02:00
Petr Štetiar
c836ca84e8 scripts/download.pl: silence can't exec curl warning
When running build in verbose mode `make V=s` we can see a lot of
following warnings when curl is not available in the system:

 Can't exec "curl": No such file or directory at scripts/download.pl line 77.

So lets fix it by redirecting of the stderr to null hole.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-13 09:14:09 +02:00
Josh Roys
90c6e3aedf
scripts: always check certificates
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.

TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"

Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-10 15:24:22 +02:00
Petr Štetiar
06e01e817e scripts: xxdi.pl: add xxd -i compat mode
So it can serve as a standalone drop in replacement for xxd utility used
currently mostly in U-Boot packages with `xxd -i` mode which outputs C
include file style, with aim for byte to byte identical output, so the
eventual difference in the generated output is easily spottable.

Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [perl-fu]
2022-09-06 08:04:53 +02:00
Jo-Philipp Wich
8b278a76d9 scripts: xxdi.pl: remove File::Slurp dependency
In order to make it more portable.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-06 08:04:53 +02:00
Petr Štetiar
2117d04a3a scripts: add xxdi.pl
xxdi.pl is a Perl script that implements vim's 'xxd -i' mode so that
packages do not have to use all of vim just to get this functionality.

References: #10555
Source: 97a6bd5cee/xxdi.pl
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-06 08:04:53 +02:00
Christian Marangi
7be01fe13b
scripts: ext-toolchain: add support for musl
Openwrt now supports only glibc and musl. Add support for musl and
rework the libc check to handle the new config flags and correctly
compile package basend on that.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-07-24 19:53:45 +02:00
Christian Marangi
75311977f5
scripts: ext-toolchain: add support for info.mk in probe_cc
Openwrt generate info.mk that contains the libc type. For probe_cc check
if the file exist and parse directly it for LIBC type.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-07-24 19:53:44 +02:00
Christian Marangi
ddeabc75eb
scripts: ext-toolchain: actually probe libc type on config generation
Currently we never call probe_cc before config generation, this cause
the script to never actually detect the correct libc type.
Call probe_cc before config generation to correctl set the .config file.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-07-24 19:53:44 +02:00
Christian Marangi
f4dd18ca39
scripts: ext-toolchain: add option to overwrite config
It can be useful to overwrite an already generated config.
Option are simply added at the end of the config and make defconfig
will overwrite the relevant option with the new one.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-07-24 19:53:44 +02:00
Christian Marangi
53c293262f
scripts: ext-toolchain: fix wrong prefix in print_config generation
The parsed prefix in print_config is wrong and this produce broken
generated .config that won't work with any external toolchain.

Currently the prefix from a CC of

'arm-openwrt-linux-muslgnueabi-gcc-12.1.0'

produce a prefix

'arm-openwrt-linux-muslgnueabi-gcc-'

This is wrong as the real prefix should be

'arm-openwrt-linux-muslgnueabi-'

This is probably caused by a change in how the toolchain is now handled
that now append also the gcc version. Probably in ancient days the
version wasn't part of the name and the prefix generation stripped the
'-gcc' instead of the gcc version.

Fix this and correctly strip the gcc version and the gcc suffix to
correctly call toolchain bins.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-07-24 19:53:43 +02:00
David Bauer
a0b7fef0ff ramips: add support for ZyXEL NWA50AX / NWA55AXE
Hardware
--------
CPU:    Mediatek MT7621
RAM:    256M DDR3
FLASH:  128M NAND
ETH:    1x Gigabit Ethernet
WiFi:   Mediatek MT7915 (2.4/5GHz 802.11ax 2x2 DBDC)
BTN:    1x Reset (NWA50AX only)
LED:    1x Multi-Color (NWA50AX only)

UART Console
------------
NWA50AX:
Available below the rubber cover next to the ethernet port.
NWA55AXE:
Available on the board when disassembling the device.

Settings: 115200 8N1

Layout:

<12V> <LAN> GND-RX-TX-VCC

Logic-Level is 3V3. Don't connect VCC to your UART adapter!

Installation Web-UI
-------------------
Upload the Factory image using the devices Web-Interface.

As the device uses a dual-image partition layout, OpenWrt can only
installed on Slot A. This requires the current active image prior
flashing the device to be on Slot B.

If the currently installed image is started from Slot A, the device will
flash OpenWrt to Slot B. OpenWrt will panic upon first boot in this case
and the device will return to the ZyXEL firmware upon next boot.

If this happens, first install a ZyXEL firmware upgrade of any version
and install OpenWrt after that.

Installation TFTP
-----------------
This installation routine is especially useful in case
 * unknown device password (NWA55AXE lacks reset button)
 * bricked device

Attach to the UART console header of the device. Interrupt the boot
procedure by pressing Enter.

The bootloader has a reduced command-set available from CLI, but more
commands can be executed by abusing the atns command.

Boot a OpenWrt initramfs image available on a TFTP server at
192.168.1.66. Rename the image to owrt.bin

 $ atnf owrt.bin
 $ atna 192.168.1.88
 $ atns "192.168.1.66; tftpboot; bootm"

Upon booting, set the booted image to the correct slot:

 $ zyxel-bootconfig /dev/mtd10 get-status
 $ zyxel-bootconfig /dev/mtd10 set-image-status 0 valid
 $ zyxel-bootconfig /dev/mtd10 set-active-image 0

Copy the OpenWrt ramboot-factory image to the device using scp.
Write the factory image to NAND and reboot the device.

 $ mtd write ramboot-factory.bin firmware
 $ reboot

Signed-off-by: David Bauer <mail@david-bauer.net>
2022-07-20 21:52:06 +02:00
Wenli Looi
efca76ffce
image: add support for Netgear encrypted image
Netgear encrypted image is used in various devices including WAX202,
WAX206, and EX6400v3. This image format also requires a dummy squashfs4
image which is added here as well.

References in WAX202 GPL source:
https://www.downloads.netgear.com/files/GPL/WAX202_V1.0.5.1_Source.rar

* openwrt/bootloader/u-boot-mt7621-2018.09-gitb178829-20200526/board/ralink/common/dual_image.c
  Bootloader code that verifies the presence of a squashfs4 image, thus
  a dummy image is added here.

* openwrt/tools/imgencoder/src/gj_enc.c
  Contains code that generates the encrypted image. There is support for
  adding an RSA signature, but it does not look like the signature is
  verified by the stock firmware or bootloader.

* openwrt/tools/imgencoder/src/imagekey.h
  Contains the encryption key and IV. It appears the same key/IV is used
  for other Netgear devices including WAX206 and EX6400v3.

Signed-off-by: Wenli Looi <wlooi@ucalgary.ca>
2022-07-19 14:49:03 +02:00
Sander Vanheule
ebfe66e494 scripts: fix CAMEO tag generator
What should have been only cosmetic changes, ended up in breaking the
script. Rename UIMAGE_CRC_SLICE back to (the original) UIMAGE_CRC_OFF.

Fixes issue #10204 "cameo-tag.py broken"

Reported-by: Markus Stockhausen <markus.stockhausen@gmx.de>
Fixes: f9e840b657 ("scripts: add CAMEO tag generator")
Signed-off-by: Sander Vanheule <sander@svanheule.net>
2022-07-05 10:18:06 +02:00
Markus Stockhausen
f9e840b657 scripts: add CAMEO tag generator
This script inserts CAMEO tags into an uImage to make U-Boot
of DGS-1210 switches happy.

Signed-off-by: Markus Stockhausen <markus.stockhausen@gmx.de>
Suggested-by: Sander Vanheule <sander@svanheule.net> # Mutual checksum algorithm
[commit title prefix, trailing whitespace, OpenWrt capitalisation, move
CRC calculation comment, use UIMAGE_NAME_*, remove parentheses for
return, use f-string instead of str()]
Signed-off-by: Sander Vanheule <sander@svanheule.net>
2022-07-05 09:55:36 +02:00
Mikhail Zhilkin
0cb1dc0277 scripts: add support for Sercomm PID
This scripts creates Sercomm PID file. PID is necessary for the factory
images creation of variuos Sercomm-based devices (Beeline, Netgear,
Etisalat).

Size: 0x70
+-------+------+---------------+------------------+
| Start | Size | Value* (ASCII)| Description      |
+=======+======+===============+==================+
| 0x0   | 0x8  | 10100         | Hardware version |
+-------+------+---------------+------------------+
| 0x8   | 0x8  | 444245 (DBE)  | Hardware ID      |
+-------+------+---------------+------------------+
| 0x64  | 0x4  | 1002          | Software version |
+-------+------+---------------+------------------+
*for Beeline Smartbox GIGA

Signed-off-by: Mikhail Zhilkin <csharper2005@gmail.com>
2022-07-03 20:25:38 +02:00
Mikhail Zhilkin
87092b3c06 scripts: add support for Sercomm kernel header
This scripts creates custom kernel header that necessary for Sercomm
mt7621 devices:
- Sercomm S3
- Beeline SmartBox Giga
- Beeline SmartBox Pro
- Beeline Smartbox Turbo
- Beeline Smartbox Turbo+
- WiFire S1500.NBN

Header format
-------------
+--------+---------------+------------------------+
| Offset | Value         | Description            |
+========+===============+========================+
| 0x0    | 53 65 72 00   | Magic "Ser."           |
+--------+---------------+------------------------+
| 0x4    | 04 00 00 01   | End offset of RootFS   |
+--------+---------------+------------------------+
|        |               | This header checksum   |
| 0x8    | d6 14 9a c1   | htonl(~crc)            |
+--------+---------------+------------------------+
| 0xc    | 02 ff ff ff   | Constant               |
+--------+---------------+------------------------+
| 0x10   | 00 01 40 00   | Kernel start offset    |
+--------+---------------+------------------------+
| 0x14   | c6 94 24 00   | Kernel length          |
+--------+---------------+------------------------+
|        |               | Kernel checksum        |
| 0x18   | e7 78 89 f1   | htonl(~crc)            |
+--------+---------------+------------------------+
| 0x1c   | 00 00 00 00   | Constant               |
+--------+---------------+------------------------+
| 0x20   | ff ff ff ff   | Constant               |
+--------+---------------+------------------------+
| 0x24   | ff ff ff ff   | Constant               |
+--------+---------------+------------------------+
| 0x28   | 00 00 00 01   | RootFS offset          |
+--------+---------------+------------------------+
|        |               | RootFS length          |
| 0x2c   | 04 00 00 00   | Always 0x4, we check   |
|        |               | UBI magic only         |
+--------+---------------+------------------------+
|        |               | RootFS checksum        |
| 0x30   | 1c fc 55 2d   | htonl(~crc)            |
|        |               | Const for UBI magic    |
+--------+---------------+------------------------+
| 0x34   | 00 00 00 00   | Constant               |
+--------+---------------+------------------------+
| 0x38   | ff ff ff ff … | Pad to 0x100           |
+--------+---------------+------------------------+

Signed-off-by: Mikhail Zhilkin <csharper2005@gmail.com>
2022-07-03 20:25:38 +02:00
Luiz Angelo Daros de Luca
2fd66e058b scripts: add cameo image header generator
The cameo header is a 0x40-byte header used by D-Link DGS 1210 switches
and Apresia ApresiaLightGS series. cameo-imghdr.py is a clean-room
reimplementation of imghdr present in the DGS-1210-28-GPL package.

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
[fix board_version argument's help text]
Signed-off-by: Sander Vanheule <sander@svanheule.net>
2022-06-28 22:20:09 +02:00
Paul Spooren
7a73221332 build: use numeric-owner in ipkg-build
To create packages the `ipkg-build` script is used which double packs
`control.tar.gz` and `data.tar.gz` to a single package. By default it's
using a verbose username instead of a numeric value for files.

Official OpenWrt images (artifacts) are created within docker containers
which do not seem to contain those verbose usernames and instead
defaults to numeric values.

This becomes a problem when rebuilding public artifacts because other
build environments may offer verbose usernames and there the created
packages is different from the official ones.

With this commit `ipkg-build` always uses numeric values for user/group
and thereby making it easier to reproduce official artifacts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-19 22:59:50 +02:00
Doug Kerr
6f692c9c49 scripts: format to black
clean up formatting with black using 80 character line limit

Signed-off-by: Doug Kerr <dek3rr@gmail.com>
2022-04-16 14:53:17 +02:00
Doug Kerr
0642a2166b scripts: use std library for jam crc32 calculation
CRC32 is available in a standard library. It seems reasonable
to defer to that rather than run a custom implementation.

Signed-off-by: Doug Kerr <dek3rr@gmail.com>
2022-04-16 14:53:17 +02:00
Daniel Golle
4d289ae7e6
scripts/gen_image_generic.sh: fix order of files in EFI bootfs
mtools recursive copy (mcopy -s ...) is using READDIR(3) to iterate
over the directory entries, hence they end up in the FAT filesystem in
traversal order which breaks reproducibility (rather than being added
to the FAT filesystem in a reproducible order). Implement recursive
copy in gen_image_generic.sh in Shell code instead, as in that way we
can force files to be copied in reproducible order.

Fixes: aece8f5ae8 ("scripts/gen_image_generic.sh: generate reproducible EFI filesystem")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-04-15 01:17:04 +01:00
Daniel Golle
aece8f5ae8
scripts/gen_image_generic.sh: generate reproducible EFI filesystem
Generate FAT filesystem for EFI boot in a reproducible way:
 * use '--invariant' option of mkfs.fat
 * set timestamps of all files to SOURCE_DATE_EPOCH
 * make sure files are ordered locale-independent

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-04-11 15:29:33 +01:00
Daniel Golle
1d77dca3b3
scripts: make sure sort-order is independent from locale
Set LC_ALL=C environment variable when calling 'sort' as the sort
order otherwise depends on the locale set.

Fixes: 56ce110b73 ("scripts: make sure conffiles are sorted")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-04-06 17:33:31 +01:00
Paul Spooren
56ce110b73 scripts: make sure conffiles are sorted
It may happen that conffiles are in different order on different builds.
Make sure they have the same order by sorting them.

FIX: #9612

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-01 13:08:58 +01:00
Paul Spooren
8822a8d850 build: store sha256_unsigned in JSON
Introduce `sha256_unsigned` which is a checksum of the image _before_ a
signature is attached. This is helpful to compare image reproducibility.

Since the `.sha256sum` file is located in the $(KDIR) folder, switch
$(BIN_DIR) with $(KDIR) to simplify the code. The value of $(BIN_DIR)
itself is not stored inside the resulting JSON file, so it can be
replaced.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-03-29 21:41:06 +01:00
Daniel Golle
068ea2cde0
scripts/gen_image_generic.sh: make ext4 bootfs reproducible
Set fixed timestamp for kernel other files in /boot filesystem.
This should help making x86 *combined* images reproducible.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-03-28 20:52:37 +01:00
Daniel Golle
e89c85249e scripts/mkits.sh: replace forgotten '-' with ${REFERENCE_CHAR}
Cosmetical change to improve style in mkits.sh.

Fixes: fd67908647 ("scripts: mkits.sh: Allow legacy @ mode for dts creation")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-03-24 00:46:45 +00:00
Rui Salvaterra
6b822189b1 toolchain: binutils: add support for version 2.38
While at it, also fix the download.xs4all.nl and add ftp.gnu.org to the GNU
mirrors, as it seems to be updated faster.

Deleted (upstreamed):
600-Close_the_file_descriptor.patch [1]

[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5a98fb7513b559e20dfebdbaa2a471afda3b4742

Signed-off-by: Rui Salvaterra <rsalvaterra@gmail.com>
2022-03-09 10:23:25 +00:00
Georgi Valkov
1324403eb6 mkhash: fix build errors on FreeBSD 13.0
be64enc, be16dec, and be32dec are declared on FreeBSD 13.0, in
/usr/include/sys/endian.h so we should not declare them.

Fixes the following error during feeds update:
staging_dir/host/bin/mkhash: No such file or directory

gcc scripts/mkhash.c
scripts/mkhash.c:111:1: error: redefinition of 'be64enc'
  111 | be64enc(void *buf, uint64_t u)

Signed-off-by: Georgi Valkov <gvalkov@abv.bg>
2022-03-05 18:01:04 +01:00
Petr Štetiar
57f7a86c68 check-toolchain-clean.sh: workaround stray rebuilds
It seems, that there are currently some unhandled corner cases in which
`.toolchain_build_ver` results in empty file and thus forcing rebuilds,
even if the toolchain was build correctly just a few moments ago. Until
proper fix is found, workaround that by checking for this corner case
and simply populate `.toolchain_build_ver` file.

While at it, improve the UX and display version mismatch, so it's more
clear what has forced the rebuild:

 "Toolchain build version changed (11.2.0-1 != ), running make targetclean"

References: https://gitlab.com/ynezz/openwrt/-/jobs/2133332533/raw
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-02-28 15:17:11 +01:00
Petr Štetiar
9116096c0f check-toolchain-clean.sh: fix shellcheck warnings
Fixes following complaints and suggestions:

 In scripts/check-toolchain-clean.sh line 2:
 eval `grep CONFIG_GCC_VERSION .config`
      ^-- SC2046 (warning): Quote this to prevent word splitting.
      ^-- SC2006 (style): Use $(...) notation instead of legacy backticks `...`.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-02-28 15:17:11 +01:00
Eneas U de Queiroz
65c28d7a67 scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built before running it, aborting in case of failure.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>[removed Fixes: due revert]
2022-02-26 13:36:30 +01:00
Petr Štetiar
d6119e2f38 Revert "scripts/diffconfig.sh: ensure config/conf is built"
This reverts commit c0849c1d9c as it seems
to introduce regression in config.buildinfo files produced at least by
buildbots:

 $ curl -s https://downloads.openwrt.org/snapshots/targets/ath79/generic/config.buildinfo | head -2
 make[3]: Entering directory '/builder/shared-workdir/build'
 make[3]: Leaving directory '/builder/shared-workdir/build'

References: https://github.com/openwrt/openwrt/issues/9297#issuecomment-1049719381
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-02-24 11:44:00 +01:00
Eneas U de Queiroz
c0849c1d9c scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built befpre running it.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-22 16:37:23 +01:00
Eneas U de Queiroz
009293c52e build: scripts/config - update to kconfig-v5.14
Functional Changes
---------- -------
- make 'imply' not impose any restrictions: allow symbols implied by y
  to become m
- change "modules" from sub-option to first-level attribute

Bugfixes
--------
- nconf: fix core dump when searching in empty menu
- nconf: stop endless search loops
- xconfig: fix content of the main widget
- xconfig: fix support for the split view mode

Other Changes
----- -------
- highlight xconfig 'comment' lines with '***'
- xconfig: navigate menus on hyperlinks
- xconfig: drop support for Qt4
- improve host ncurses detection

Update the 'option modules' usage to just 'modules' in Config.in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-19 13:10:01 +01:00
Paul Spooren
7fae1e5677 feeds: use git-src-full to allow Git versioning
Both $(AUTORELEASE) and $(PKG_SRC_VERSION) (from luci.git) use the Git
log to determine releases and package timestamps.

Feeds are shallow cloned by default, resulting in an incomplete Git log
and therefore different local package versions than offered upstream.

This commits sets the default feeds to use `src-git-full` to solve that.

Add fixes from "2b1d92f: scripts/feeds: silence git warning by selecting
pull style" to `src-git-full`

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-02-15 00:24:24 +01:00
Paul Spooren
2120cad38d build: store source_date_epoch as integer
The value is retreived from a env variable which defaults to be read as
a string. However the SOURCE_DATE_EPOCH is a unix timestamp aka integer.

Fix this to allow downstream tools to parse the value directly.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-02-14 14:40:34 +01:00
Felix Fietkau
ebc36ebb23 scripts/feeds: install targets to target/linux/feeds and support overriding
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-01-24 13:29:05 +01:00
Felix Fietkau
0bdf8d1206 scripts/feeds: fix installing targets without explicitly specifying the feed
Add similar code to what is done on packages

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-01-24 13:29:05 +01:00
Ansuel Smith
88204bfa82 treewide: drop use of which
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-01-17 09:14:26 +01:00
Paul Spooren
02de391b08 scripts: dump-target-info print kernel versions
This commits adds the ability to print Kernel versions of all
targets/subtargets. If a testing Kernel is set print that version as
well.

Example output:

    apm821xx/nand 5.10
    apm821xx/sata 5.10
    arc770/generic 5.4
    archs38/generic 5.4
    armvirt/32 5.10
    armvirt/64 5.10
    at91/sam9x 5.10
    at91/sama5 5.10
    ath25/generic 5.4
    ath79/generic 5.4 5.10
    ath79/mikrotik 5.4 5.10
    --- %< ---

This should help to get a quick update on the state of Kernels.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-12-29 14:06:10 +01:00
Enrico Mioso
57c1f3f9c5
base-files: add eMMC sysupgrade support
Adds generic support for sysupgrading on eMMC-based devices.

Provide function emmc_do_upgrade and emmc_copy_config to be used in
/lib/upgrade/platform.sh instead of redundantly implementing the same
logic over and over again.
Similar to generic sysupgrade on NAND, use environment variables
CI_KERNPART, CI_ROOTPART and newly introduce CI_DATAPART to indicate
GPT partition names to be used. On devices with more than one MMC
block device, CI_ROOTDEV can be used to specify the MMC device for
partition name lookups.

Also allow to select block devices directly using EMMC_KERN_DEV,
EMMC_ROOT_DEV and EMMC_DATA_DEV, as using GPT partition names is not
always an option (e.g. when forced to use MBR).

To easily handle writing kernel and rootfs make use of sysupgrade.tar
format convention which is also already used for generic NAND support.

Signed-off-by: Enrico Mioso <mrkiko.rs@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
CC: Li Zhang <li.zhang@gl-inet.com>
CC: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2021-12-02 20:42:58 +00:00
Robert Marko
5a305e429f scripts: ubinize-image: make rootfs optional
Currently ubinize-image script always expects the
rootfs image to be passed and a volume for it created.

So, to allow only ubinizing a kernel for example which
the MikroTik hAP ac3 and other new NAND devices from
MikroTik require make rootfs an optional parameter like
kernel.

Signed-off-by: Robert Marko <robimarko@gmail.com>
2021-11-28 17:17:22 +01:00
Felix Fietkau
db34b93331 toolchain: add a version that can be bumped to force toolchain/target rebuild
This can be used to ensure that the compiled code is up to date, when
something important changes in the toolchain.
A recent example of this is the gcc 11 fix for a code miscompilation issue

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-16 22:02:45 +01:00
Arne Zachlod
b5ab6d0bc5 scripts/env: fix env for git conf init.defaultBranch not set to "master"
Since version 2.28, git has a config option init.defaultBranch to set the name
of the first branch created with git init. The env script expects this name to
be "master". This commit sets the initial branch name to "master"
instead of using the git configured one.

Signed-off-by: Arne Zachlod <arne@nerdkeller.org>
2021-11-13 10:29:41 -10:00
Josh Soref
08622de7d6 scripts: fix various typos
This only affects typos in comments or user-facing output.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
[only picks changes to scripts, drop "commandline" replacement,
 fix case for "arbitrary", improve commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-10-31 21:24:47 +01:00
Jan-Niklas Burfeind
88a2ea41da scripts: eva_ramboot.py: remove unused import
concludes:
commit e7bc8984d9 ("scripts: make eva_ramboot.py offset configurable")

Signed-off-by: Jan-Niklas Burfeind <git@aiyionpri.me>
2021-10-30 15:00:22 +02:00
Daniel Golle
b2aca61360
base-files, metadata: support additional group membership
Some packages may require additional group membership for the system
user added by that package. Allow defining additional groups as third
member of the ':'-separated tuple, allowing to specify multiple
','-separated groups with optional GID.

Example:
USERID:=foouser=1000:foogroup=1000:addg1=1001,addg2=1002,addg3

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-10-28 16:27:21 +01:00
Hans Dedecker
abc7a97e9c Revert "ethtool: update to v5.14"
This reverts commit 7630001427

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
2021-10-23 21:39:00 +02:00
Hans Dedecker
7630001427 ethtool: update to v5.14
Update to newly released version 5.14

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
2021-10-23 21:35:38 +02:00
Felix Fietkau
b45ff2d023 scripts/rstrip.sh: do not strip .o files with STRIP_KMOD
Fixes packaging of eBPF modules

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-10-11 11:49:14 +02:00
Paul Spooren
3128dfc18a scripts: package-metadata add pkgmanifestjson call
The new `pkgmanifestjson` call prints all package manifest of a feed in
JSON format. This function can be used to print an overview of packages
information used for downstream tooling.

The script is entirely based on Petrs work on dependency visualisation.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-23 21:57:22 -10:00
Paul Spooren
108901fffb scripts: store maintainer in package metadata
The maintainer could be usable for downstream tooling, so start storing
it in the metadata.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-23 21:38:59 -10:00
Damien Mascord
fd67908647 scripts: mkits.sh: Allow legacy @ mode for dts creation
commit 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
broke support for Meraki MR32 and this patch makes the replacement
configurable allowing for specifying the @ or - or whatever character
that is desired to retain backwards compatibility with existing devices.

For example, this patch includes the fix for the Meraki MR32 in
target/linux/bcm53xx/image for meraki_mr32:

  DEVICE_DTS_DELIMITER := @
  DEVICE_DTS_CONFIG := config@1

Fixes: 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
Signed-off-by: Damien Mascord <tusker@tusker.org>
[Added tags, checkpatch.pl fixes, noted that this is for old stuff]
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2021-09-23 20:49:32 +02:00
Ansuel Smith
93f488fc37 scripts: handle gcc and linux in dl_cleanup script
Handle gcc and linux with a special regex that set their progname with
their major version. This way every minor version can be cleared. The
build cleanup logic can be tweaked later to clean the entire toolchain
and target dir with a different gcc version.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[reformat commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-22 22:24:39 -10:00
Ansuel Smith
ff875876da scripts: format dl_cleanup to black format python style
Forma dl_cleanup python script to black style.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-09-22 19:54:00 -10:00
Ansuel Smith
4eb4c3c469 scripts: add missing regex for dl_cleanup script
Regex xxx-YYYY-MM-DD-GIT_SHASUM was missing. Add the new regex to improve
and better find outdated package. This also fix a bug where some bug were
incorrectly detected as packagename-yyyy-mm-dd instead of packagename due
to them be parsed by the wrong parser

Example:

    openwrt-keyring-2021-02-20-49283916.tar.xz

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[added example in commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-22 19:51:44 -10:00
Paul Spooren
5ef4608c02 build: store artifacts in JSON
Multiple profiles create artifacts, these should be stored in the JSON
file as well, allowing downstream tooling to show those files, too.

Artifacts don't have specific filesystems so only the fields `name`,
`type` and `sha256` are available.

Rename env variable names from IMAGE_ to FILE_ prefixes to reflect that
images, kernels and artifacts are added with the same command.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-09-14 10:42:38 -10:00
Daniel Golle
98bccdafd7
base-files: rename 'sdcard' to 'legacy-sdcard'
While an image layout based on MBR and 'bootfs' partition may be easy
to understand for users who are very used to the IBM PC and always have
the option to access the SD card outside of the device (and hence don't
really depend on other recovery methods or dual-boot), in my opinion
it's a dead end for many desirable features on embedded systems,
especially when managed remotely (and hence without an easy option to
access the SD card using another device in case things go wrong, for
example).

Let me explain:

* using a MSDOS/VFAT filesystem to store kernel(s) is problematic, as a
  single corruption of the bootfs can render the system into a state
  that it no longer boots at all. This makes dual-boot useless, or at
  least very tedious to setup with then 2 independent boot partitions
  to avoid the single point of failure on a "hot" block (the FAT index
  of the boot partition, written every time a file is changed in
  bootfs). And well: most targets even store the bootloader environment
  in a file in that very same FAT filesystem, hence it cannot be used
  to script a reliable dual-boot method (as loading the environment
  itself will already fail if the filesystem is corrupted).

* loading the kernel uImage from bootfs and using rootfs inside an
  additional partition means the bootloader can only validate the
  kernel -- if rootfs is broken or corrupted, this can lead to a reboot
  loop, which is often a quite costly thing to happen in terms of
  hardware lifetime.

* imitating MBR-boot behavior with a FAT-formatted bootfs partition
  (like IBM PC in the 80s and 90s) is just one of many choices on
  embedded targets. There are much better options with modern U-Boot
  (which is what we use and build from source for all targets booting
  off SD cards), see examples in mediatek/mt7622 and mediatek/mt7623.

Hence rename the 'sdcard' feature to 'legacy-sdcard', and prefix
functions with 'legacy_sdcard_' instead of 'sdcard_'.

Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-08-16 12:22:17 +01:00
Stijn Tintel
8347d8b4fb base-files: add generic sdcard upgrade method
Add a generic sdcard upgrade method instead of duplicating code in yet
another target, and add a feature flag to only install this upgrade
method in targets that set this flag. Copied from mvebu.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:34:40 +03:00
Stijn Tintel
84c9570aaa scripts/target-metadata.pl: order features alphabetically
We generaly do this in many other places, so let's do it here also.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:30:37 +03:00
Yonghyu Ban
7270d768da scripts/mkits.sh: Fix the hash algorithm paramter
The mkits.sh script help message states hash algorithm can be
specified using the -H command-line option, but it does not work
currently due to a bug in the script.

This patch fixes this problem by changing the option from -S to
-H and specify getopts parameter after it

Signed-off-by: Yonghyu Ban <yonghyu@empo.im>
2021-07-18 19:37:30 +01:00
Ansuel Smith
64ce35bfaf scripts: check if dl directory exist in dl_cleanup script
Check if the provided dl directory exist and return on error.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-07-05 14:15:58 -10:00
Paul Spooren
fd0d9909bf build,json: fix generation with empty profiles
If the image generation doesn't add any profiles to the output the
*profile merge* will fail. To avoid that set an empty profile as
fallback.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-20 22:42:29 -10:00
Moritz Warning
a463b96241 build: preserve profiles.json between builds
Keep other profiles.json content if the data belongs to the current
build version.

Also useful for the ImageBuilder, which builds for a single model each
time. Without this commit the profiles.json would only contain the
latest build profile information.

Signed-off-by: Moritz Warning <moritzwarning@web.de>
[improve commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-16 21:28:53 -10:00
Karel Kočí
1aa80ce393 scripts/feeds: generate index after all feeds are updated
This separates index update from feed update. The result is that all
requested feeds are first updated and only then indexed.

The reason for this change is to prevent errors being reported and
potentially invalid index being generated thanks to cross feeds
dependency.
The feeds script pulls in default all feeds as they come and on install
prefers packages from first feeds (unless special feed is requested).
Thus order of feeds in some way specifies preferences. This is handy for
downstream distributions as they can simply override any package from
upstream feeds by placing their feed before them. This removes need to
patch or fork upstream feeds.
The problem is that such feed most likely depends in some way also on
subsequent feeds. The most likely feeds are 'packages' or 'luci'. The
example would be Python package that needs 'python.mk' from 'packages'
feed. Ordering custom feed after dependent feeds is sometimes just not
possible because of preference requirement described before.
The solution is to just first pull all feeds and generate indexes only
after that. In the end this ensures that index is generated correctly at
first try without any error.

In terms of code this removes 'perform_update' argument from
'update_feed' as with index update removal the update is the only action
performed in that subroutine. Thus this moves condition to 'update'
subroutine.

Signed-off-by: Karel Kočí <karel.koci@nic.cz>
2021-06-07 12:44:50 -10:00
Paul Spooren
ec4852f731 scripts: config.guess: update to 2021-05-24
This script hasn't seen an update in multiple years, update it to the
latest version provided upstream. Both `config.guess` and `config.sub`
are copied from upstream[1] and not modified.

The full changelog is available within the upstream repository[1].

[1]: https://git.savannah.gnu.org/git/config.git

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-05-29 13:03:07 +02:00
Leonardo Mörlein
b993b68b6c build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
2021-05-13 15:13:15 +02:00
Paul Spooren
7880a64848 build,json: 3rd fixup of default_packages
This became a bit of a tragedy, caused by a corner cases which wasn't
put into account during testing. DEFAULT_PACKAGES are defined in
target/linux/<target>/Makefile but also in
target/linux/<target>/<subtarget>/target.mk.

The latter was no longer imported when using DUMP=1, however not using
DUMP=1 while running the Makefile in target/linux/<target>/ caused duplicate
packages in the list.

As a solution, which should have been used from day 0, `make` runs in
target/linux/ without DUMP=1, resulting in no duplicate packages and all
inclusions from include/target.mk, linux/target/<target>/{Makefile,
<subtarget>/target.mk}

While at it, sort the list of default packages.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-25 10:25:54 -10:00
Paul Spooren
b36068d35d build,json: fixup fixup of arch_packages
The commit "1bf2b3fe90 build,json: fixup missing arch_packages" fixes
the missing package architecture locally but runs $(TOPDIR)/Makefile
rather than a target specific one. While this works on local builds just
fine, it causes the buildbots to add garbage to the `arch_packages`
variable:

    cd \"/builder/shared-workdir/build\"; git log --format=%h -1
    toolchain > /builder/shared-workdir/build/tmp/.ver_check\ncmp -s
    /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check
    || { \\\n\trm -rf
    /builder/shared-workdir/build/build_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl
    /builder/shared-workdir/build/build_dir/toolchain-x86_64_gcc-8.4.0_musl;
    \\\n\tmkdir -p
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp;
    \\\n\tmv /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check;
    \\\n}\nx86_64

Only the last line contains the desired string.

Future investigation should check why the build system prints this to
stdout rather than stderr.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 19:03:24 -10:00
Paul Spooren
1bf2b3fe90 build,json: fixup missing arch_packages
Fix 7f4c2b1 "build,json: fix duplicates in default_packages" which
removed duplicate default packages but also removed the package
architecture from the profiles.json.

If DUMP=1 is set, the `ARCH_PACKAGES` is no longer exported and
therefore empty. Fix this by running make twice, once with DUMP=1 and
once without.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 13:41:34 -10:00
Paul Spooren
7f4c2b1a4f build,json: fix duplicates in default_packages
Calling without the DUMP=1 argument causes the target specific Makefile
to be "included" again which adds the target specific packages twice,
once on the actual run and once included from `include/target.mk`.

This led to duplicate package entries, causing confusion in downstream
projects using the generated JSON files.

While at it, apply `black` style to Python script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-20 23:58:52 -10:00
Daniel Golle
6890f6fe13 include/image*: add support for device-tree overlays
Add new target feature 'dt-overlay' which makes DTC keep the symbol
names in the generated dtb.
Make sure additional DT overlay sources specified by the new device
variable DEVICE_DTS_OVERLAY get compiled together with the main DTS
(currently overlays got to be in the same folder). Let Build/fit pass
the generated DT overlay blobs to mkits.sh.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:45:34 +00:00
Daniel Golle
4e6de4f093 scripts/mkits.sh: add support for adding DT overlay blobs to image
Allow adding multiple device tree overlay blobs to an image and
generate configurations for each of them.
This is useful on boards with modern U-Boot which allow e.g. user-
configurable peripherals ("shields") in that way.
Note that currently, each generated configuration adds exactly one
overlay on top of the base image, ie. adding multiple overlays at the
same time is not yet supported.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:41:24 +00:00
Paul Spooren
bb95be9265 scripts,ipkg-build: use realpath for pkg_dir
This allows manual execution of the ipkg-build script even with
releative path.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:32:30 -10:00
Paul Spooren
72f481b157 scripts,ipkg-build: apply shellcheck
This commit cleans the `ipkg-build` script via changes suggested by
shellcheck. These are mostly word splitting issues.

Remove the definition of GZIP, this adds three "lookups" of the `gzip`
binary but the rest of the build system doesn't seem to use such
improvements neither.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:31:43 -10:00
Robert Marko
5ec60cbe9d scripts: mkits.sh: replace @ with - in nodes
U-boot will reject the nodes with @ for the address since
commit:
79af75f777

This in turn will cause the failure to boot with OpenWrt
generated images.

So, to rectify that simply replace @ with -.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cover also newly added rootfs@1 and initrd@1 nodes)
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
2021-03-05 15:45:33 +00:00