Commit Graph

850 Commits

Author SHA1 Message Date
Paul Spooren
1fdf6b745c treewide: replace which with command -v
Fix shellcheck SC2230
> which is non-standard. Use builtin 'command -v' instead.

Using `command -v` is POSIX compliant while `which` is not.  Also to
mention, `command -v` is a shell builtin whereas `which` is a separate
busybox applet.

Once applied to everything concerning OpenWrt we can disable the busybox
feature `which` and save 3.8kB.

Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[also replace cases in zram-swap]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-12 11:08:05 +02:00
Petr Štetiar
80d360202f scripts: remove checkpatch.sh
That file was added accidentally in v3 of the patch and I haven't
properly reviewed that patch before pushing it.

Fixes: 656b562aff ("scripts: Add Buildbot dump-target-info.pl script")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-08-04 14:25:44 +02:00
Paul Spooren
656b562aff scripts: Add Buildbot dump-target-info.pl script
The script comes from buildbot.git[0] and is used to print available
targets and architectures, which are then build.

As the buildbot clones openwrt.git anyway, the script might as well live
here to be used for other cases as well, e.g. determining what
architectures are available when building Docker containers or show
developers an overview which architectures are used by which target.

It's called with either the parameter `architectures` or `targets`,
showing architectures followed by supported targets or targets, followed
by the supported architectures:

$ ./scripts/dump-target-info.pl architectures
aarch64_cortex-a53 bcm27xx/bcm2710 mediatek/mt7622 mvebu/cortexa53 sunxi/cortexa53
aarch64_cortex-a72 bcm27xx/bcm2711 mvebu/cortexa72
...

$ ./scripts/dump-target-info.pl targets
apm821xx/nand powerpc_464fp
apm821xx/sata powerpc_464fp
...

In the future the the script could be removed from the buildbot
repository and maintained only here.

Rename `dumpinfo.pl` to `dump-target-info.pl` to improve verbosity of
filename.

[0]: https://git.openwrt.org/?p=buildbot.git;a=blob;f=scripts/dumpinfo.pl;h=aa97f8d60379076a41b968402e9337cea824ece5;hb=HEAD

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-04 09:28:39 +02:00
Adrian Schmutzler
b99623329c scripts/checkpatch.pl: fix README.md file name after rename
checkpatch.pl uses a list of files to detect the root OpenWrt
directory. This includes README, which has been renamed to
README.md in the previous commit.

Update the file name in checkpatch.pl to prevent errors like the
following when running the script:

   Must be run from the top-level dir. of a OpenWrt tree

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-02 15:51:39 +02:00
Adrian Schmutzler
1d5260cf72 build: add option to mark devices as BROKEN
By specifying "BROKEN := 1" or "BROKEN := y" for a device, it will be
hidden (and deselected) by default. By that, it provides a stronger
option to "disable" a device beyond just using DEFAULT := n.

To make these devices visible, just enable the BROKEN option in
developer settings as already implemented for targets and packages.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-30 21:49:07 +02:00
Paul Spooren
5b8b42c78b build,json: fix compatibility with Python 3.5
The f-string feature was introduced in Python 3.6. As Buildbots may run
on Debian 9, which comes per default with Python 3.5, this would cause
an issue. Instead of f-strings use the *legacy* `.format()` function.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-13 22:45:04 +02:00
Kevin Darbyshire-Bryant
8242c6de29 scripts/env: Fix 56f813674a scripts/env: use command -v instead of which
We don't need to see how git will be executed and it produces non silent
output on 'scripts/env diff' commands when there are no differences
unlike before.

Re-introduce original silent behaviour.

Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
2020-07-12 13:29:28 +01:00
Adrian Schmutzler
52cdd6185e scripts/mkits.sh: fix use of printf
Due to a line break, printf was accidentally called with three
arguments instead of two, causing a different output than before.

Fix it by splitting the printf command into two lines.

Fixes: 907053193a ("scripts/mkits.sh: replace echo -e with printf")

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-07-11 18:46:53 +02:00
Petr Štetiar
dec9ae6d9e scripts/mkits.sh: fix remaining shellcheck warning
Fixes following shellcheck warning:

 In scripts/mkits.sh line 19:
 		 "-k kernel [-D name -d dtb] -o its_file" "$(basename $0)"
                                                                       ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 14:42:32 +02:00
Rosen Penev
3d418dd51e scripts/mkits.sh: switch from bash to sh
This no longer needs bash.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
7aefc85cab scripts/mkits.sh: fix improper string and array concatenation
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
765858f7ad scripts/mkits.sh: add missing quotes
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Rosen Penev
907053193a scripts/mkits.sh: replace echo -e with printf
echo flags are not POSIX. printf does the same with added \n.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 14:42:32 +02:00
Petr Štetiar
92d7cedcad scripts/gen_image_generic.sh: fix more shellcheck warnings
Fixes following shellcheck warnings:

 In scripts/gen_image_generic.sh line 20:
 cyl=$(( (KERNELSIZE + ROOTFSSIZE) * 1024 * 1024 / (head * sect * 512)))
 ^-^ SC2034: cyl appears unused. Verify use (or export if used externally).
 --
 In scripts/gen_image_generic.sh line 34:
     [ -n "$PADDING" ] && dd if=/dev/zero of="$OUTPUT" bs=512 seek="$(($ROOTFSOFFSET + $ROOTFSSIZE))" conv=notrunc count="$sect"
                                                                       ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.
 --
 In scripts/gen_image_generic.sh line 35:
     mkfs.fat -n kernel -C "$OUTPUT.kernel" -S 512 "$(($KERNELSIZE / 1024))"
                                                       ^---------^ SC2004: $/${} is unnecessary on arithmetic variables.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 14:42:32 +02:00
Rosen Penev
aee169b06d scripts/gen_image_generic.sh: use /bin/sh
This has nothing bash specific.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
3819337cdf scripts/gen_image_generic.sh: replace -o with if/&&
-o is not well defined.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Petr Štetiar
227e47b3ca scripts/env: fix remaining shellcheck warning
Fixes following shellcheck warning:

 In scripts/env line 25:
 	exit ${1:-1}
             ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-11 13:33:28 +02:00
Rosen Penev
5b7cace9e3 scripts/env: use read -r instead of read
read mangles backslashes.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
f80a540dbf scripts/env: exit in case of failure to cd
Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
e68810ba53 scripts/env: replace \! with !
The latter is more standard. The former throws an error under
shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Rosen Penev
56f813674a scripts/env: use command -v instead of which
Simpler and built in to the shell.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-07-11 13:33:28 +02:00
Huangbin Zhan
3caad5150c build: fix compatibility with python 3.6
On a system python3 is linked to python3.6, fail to perform json_overview_image_info
 and got `TypeError: __init__() got an unexpected keyword argument 'capture_output'`.
This patch emulate the behaviour on python 3.7+.

Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
2020-07-07 19:47:24 +02:00
Sergio E. Nemirowski
ab26022cc1 build: mconf readme update
This updates mconf.c readme message to maintain less changes with
upstream and consistency with nconf.c

Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
2020-07-07 19:47:24 +02:00
Sergio E. Nemirowski
8fb0b15efd build: add nconfig
Add support for make target nconfig (ncurses)

Reviewed-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Sergio E. Nemirowski <sergio@outerface.net>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-07-07 19:47:24 +02:00
Daniel Golle
3b0f698760 build,json: fix build failure in case no data is found
Only collect arch_packages if actually generating any output.

Fixes: commit f09b9319 ("build,json: store arch_packages in profiles.json"(
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-03 22:09:24 +01:00
Paul Spooren
f09b9319c6 build,json: store arch_packages in profiles.json
The `arch_packages` contains the supported package architecture.

Previously it was necessary to parse the `Packages` index for the line
`Architecture:`, requiring both an additional parser and file download.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-07-03 15:28:05 +01:00
Paul Spooren
263f7e5bbd build: store default/device packages in JSON
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.

This information is required for downstream projects like the various
web-based interactive firmware generators.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-06-30 23:21:10 +01:00
Álvaro Fernández Rojas
3f8e163e01 scripts: config: remove accidentally added file
Also ignore the file.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 18:17:31 +02:00
Álvaro Fernández Rojas
35d7f92c60 scripts: support Sercomm load tags
Header consists in Sercomm PID bytes, followed by a SHA256 hash of the
input binary.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 13:06:42 +02:00
Álvaro Fernández Rojas
7f1250a7fe scripts: support Sercomm crypto
Sercomm firmwares are encrypted with AES 256 CBC.
The key is generated with a custom algorithm from the firmware tag:
	char key[32];
	char version[32];
	char iv[32];
	char random[32];
	char size[32];
Key must be generated with Sercomm's algorithm. However, the rest of the
header can be empty. IV and random are set to 0 on purpose.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 08:37:54 +02:00
Álvaro Fernández Rojas
58bdbe7321 scripts: support Sercomm partition tags
Sercomm uses a custom layout for partition tags:
	char part_name[32];
	char size[32];
	char part_version[32];
	char reserved[32];
	char rootfs_version[32];

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-06-02 08:33:11 +02:00
Rosen Penev
69f8983af4 scripts/flashing/flash.sh: remove trailing whitespaces
Signed-off-by: Rosen Penev <rosenp@gmail.com>
[cut out of patch with different subject]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-05-24 14:44:00 +02:00
Rosen Penev
34a2780395 scripts/flashing/flash.sh: Add missing quotes
These are in if statements (where they should be), and echo output that is
not piped to anything. These should be safe.

Found with shellcheck.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-05-24 14:44:00 +02:00
Álvaro Fernández Rojas
d90828411a scripts: add CFE Partition Tags support
Some BCM63xx NAND devices require a specific JFFS2 partition tag to verify
the JFFS2 partition validity:
	u32 part_id;
	u32 part_size;
	u16 flags;
	char part_name[33];
	char part_version[21];
	u32 part_crc32;

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-21 19:11:24 +02:00
Eneas U de Queiroz
80b350f528 build: have config-clean deal with old temp files
This is a temporary commit to have 'make config-clean' remove
temporary files from the previous scripts/config version.

The .gitignore file is updated to deal with the old files as well.

Cc: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-05-20 17:03:45 +02:00
Eneas U de Queiroz
9f843b1d43 build: scripts/config - update to kconfig-v5.6
Major changes include:
 - Much more readable reverse dependencies separated in groups
 - Improved recursive dependency report
 - More readable .config files: add comments to signal end of menus
 - More warnings for incorrect Config.in entries, such as a 'choice'
   default not contained in the 'choice'
 - Hability to properly display pseudographics with non-latin locales
 - Recursive dependencies can optionally be treated as errors

Changes from failed dcf3e63a35 attempt:
 - Recursive dependencies are treated as warnings by default
 - The option to treat them as errors is implemented as a command-line
   flag to scripts/config/conf instead of a compile-time definition
 - fixed handling of select with umnet dependencies

Cc: Petr Štetiar <ynezz@true.cz>
Cc: Jo-Philip Wich <jow@mein.io>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-05-20 17:03:45 +02:00
Álvaro Fernández Rojas
258cb7f169 scripts: cfe-wfi-tag: fix shebang
Fixes: 72985233a6 ("scripts: support CFE WFI images")
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-19 08:06:29 +02:00
Álvaro Fernández Rojas
30fcb62840 scripts: cfe-bin-header: fix shebang
Fixes: e575a7f777 ("scripts: support tags for CFE binaries")
Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-19 08:05:22 +02:00
Álvaro Fernández Rojas
72985233a6 scripts: support CFE WFI images
CFE WFI (Whole Flash Image) consists in 20 byte footer:
- u32: CRC32
- u32: WFI Version
- u32: Chip ID
- u32: Flash Type
- u32: Flags

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-18 18:24:06 +02:00
Álvaro Fernández Rojas
e575a7f777 scripts: support tags for CFE binaries
CFE loads binaries based on a 12 byte header which corresponds to:
- u32: Load Address
- u32: Entry Address
- u32: Size

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2020-05-18 18:24:06 +02:00
Paul Spooren
c737a9ee6a scripts/download: add sources CDN as first mirror
OpenWrt now has a CDN for sources at sources.cdn.openwrt.org which
mirrors sources.openwrt.org.

Downloading sources outside Europe or US (mainland) could
result in low throughput, extremely slowing down the first compilation of
the build system.

This patch adds sources.cdn.openwrt.org as the first mirror to offer
worldwide fast download speeds by default. If the CDN goes down for
whatever reason, the script jumps to the next available mirror and
downloads requested files as before (in regional varying speed).

Signed-off-by: Paul Spooren <mail@aparcar.org>
Acked-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-14 23:16:55 +01:00
Paul Spooren
14cbd8fb2d scripts: JSON merge don't crash if no JSON found
The JSON `WORK_DIR` ($(KDIR)/json_info_files) is only created if the new
image generation methods from `image.mk` are used. However some targets
like `armvirt` do not use it yet, so the folder is never created.

The `json_overview_image_info.py` script used to raise an error if the
given `WORK_DIR` isn't a folder, however it should just notify about
missing JSON files.

This patch removes the Python assert and exists with code 0 even if no
JSON files were found, as this is not necessarily an error but simply
not yet implemented. Using `glob` on an not existing `Path` results in
an empty list, therefore the for loop won't run.

Signed-off-by: Paul Spooren <mail@aparcar.org>
CC: Petr Štetiar <ynezz@true.cz>
2020-04-14 23:16:55 +01:00
Jo-Philipp Wich
7b1d809a8d Revert "build: scripts/config - update to kconfig-v5.6"
This reverts commit dcf3e63a35.

The kconfig update requires further testing and refinement until it can
remain in tree. Main problems are:

 - Recursive deps are now fatal instead of a warning
 - Previously legal syntax now leads to hard failures
 - It fails all package builds since multiple days

The updated kconfig implementation needs to cope with the current status
quo in the various package feeds before we can reconsider it for master.

It is not desirable that single broken packages can hard-fail the entire
build pipeline.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:47 +02:00
Jo-Philipp Wich
6c16d64b2a Revert "build: add option to warn on recursive dependency"
This reverts commit 3204430e38.

Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:30 +02:00
Jo-Philipp Wich
fbc01f2a4c Revert "build: config: allow bool to select a module pkg"
This reverts commit 8514b6b42c.

Reverting this commit in preparation for reverting
dcf3e63a35 ("build: scripts/config - update to kconfig-v5.6") which
introduces various unaddressed build breakages.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-04-12 00:10:16 +02:00
Eneas U de Queiroz
8514b6b42c build: config: allow bool to select a module pkg
kconfig-v5.6 disallowed a bool symbol to select another symbol that
'depends on m' (i.e. can be only 'm' on 'n').  It is, in fact, an unmet
dependency to have set to 'y'.  However, openwrt depends on the previous
behavior, to be able to build a package that can be a module or built-in
depending on a bool config.  This restores the previous behavior.

Ref: https://forum.openwrt.org/t/wireless-fails-on-snapshot-r12900-kernel-5-4-on-c2600-ipq806x
Tested-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [added forum ref]
2020-04-11 19:06:26 +02:00
Eneas U de Queiroz
3204430e38 build: add option to warn on recursive dependency
This addes the option to treat recursive dependencies as warnings
instead of errors, by running make with WARN_RECURSIVE_DEP=1.

Note that the script/config targets will not get rebuilt when you add or
remove WARN_RECURSIVE_DEP while running make.  One must run
'make config-clean' before building config with a different setting.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Eneas U de Queiroz
dcf3e63a35 build: scripts/config - update to kconfig-v5.6
Major changes include:
 - Much more readable reverse dependencies separated in groups
 - Improved recursive dependency report
 - More readable .config files: add comments to signal end of menus
 - More warnings for incorrect Config.in entries, such as a 'choice'
   default not contained in the 'choice'
 - Hability to properly display pseudographics with non-latin locales
 - Recursive dependencies are now treated as errors - this should make
   it harder for them to creep in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Eneas U de Queiroz
8636a17214 build: simplify building *config targets
Instead of passing pkg-config location through a variable when building
qconf (make xconfig), prepend its parent directory to the PATH, as it is
being done for other conf targets.

Use a Makefile pattern rule to group all 'scripts/config/%onf'
(currently conf, mconf, qconf) targets in a single rule.  Add -O2 to
CFLAGS when building them as well.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2020-04-09 12:51:10 +02:00
Paul Spooren
07449f692c build: refactor JSON info files to profiles.json
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").

They are useful for firmware wizards and script checking for
reproducibility.

Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.

This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.

Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.

Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.

As before, this creation is enabled by default only if `BUILDBOT` is set.

Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2020-04-03 12:17:45 +02:00