Commit Graph

123 Commits

Author SHA1 Message Date
Linus Walleij
f2a8763587 uboot-bcm53xx: Add U-Boot for NorthStar BCM53xx
I recently added support for the NorthStar ARM BCM53xx SoCs
to the upstream U-Boot. This is a back port on top of the
2023.04 version already imported to OpenWrt with the 5 necessary
upstream patches.

This is needed to create a small U-Boot for the BCM53xx-based
D-Link DIR-890L and I think also the DIR-885L, so that a
recent (bigger) kernel can be loaded and executed from the
SEAMA partitions on these devices.

Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
2023-08-20 18:08:13 +02:00
Christian Marangi
881235c713
CI: provide new required secret for S3 endpoint and bucket name
Provide new required secret for S3 endpoint and bucket name to permit an
easier migration to new services.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-08-05 01:58:29 +02:00
Christian Marangi
f98dc5aa43
CI: generilize S3 secret keys name and rename to proper name
Generilize S3 secret keys and rename to make them not platform specific.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-08-05 01:57:47 +02:00
Christian Marangi
14293dd901
CI: drop unused reusable workflow and dockerfiles
Drop unused reusable workflow and dockerfiles now that we moved them to
a dedicated repository.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-06-25 02:08:56 +02:00
Christian Marangi
38cc09165f
CI: migrate each workflow to use reusable workflow from dedicated repo
Migrate each workflow to use reusable workflow from dedicated repo to
skip pushing CI related commits to openwrt and better track versioning
of CI workflow.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-06-25 02:08:56 +02:00
Robert Marko
f02f6aaa8d
ipq807x: rename target to qualcommax
Currently, ipq807x only covers Qualcomm IPQ807x SoC-s.
However, Qualcomm also has IPQ60xx and IPQ50xx SoC-s under the AX WiSoC-s
and they share a lot of stuff with IPQ807x, especially IPQ60xx so to avoid
duplicating kernel patches and everything lets make a common target with
per SoC subtargets.

Start doing that by renaming ipq807x to qualcommax so that dependencies
on ipq807x target can be updated.

Signed-off-by: Robert Marko <robimarko@gmail.com>
2023-06-16 11:11:08 +02:00
Christian Marangi
de9955a62f
CI: build-tools: build all host tools
Now that we build also core packages, we need more host tools. Compile
all of them to reduce compile time on other actions.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-06-16 11:09:12 +02:00
Piotr Dymacz
92b8b18c26 CI: labeler: add sifiveu target
Add support for 'sifiveu' target and its specific packages in labeler.

Signed-off-by: Piotr Dymacz <pepe2k@gmail.com>
2023-06-12 21:22:52 +02:00
Mathew McBride
3df01b1aa4
CI: change armvirt reference to armsr
The armvirt target has been renamed to armsr.

Signed-off-by: Mathew McBride <matt@traverse.com.au>
2023-06-10 21:30:25 +02:00
Christian Marangi
218deba503
CI: label-kernel: support compile testing kernel version and all target
Add support to label-kernel for compiling testing kernel version and
check patches. To trigger this special build appent :testing to the
normal label.
Example:

- ci:kernel:ipq806x:generic:testing

Test will fail if the requested target doesn't have a defined kernel
testing version.

Also add support for testing all target and subtarget. To trigger this
some special pattern are added:
- ci:kernel:all:all
  Trigger test for all target and subtarget

- ci:kernel:all:first
  Trigger test for all target and the first subtarget in alphabetical
  order for the target.

With these special case :testing can also be used and every target and
subtarget that supports kernel testing version will be selected:
- ci:kernel:all:all:testing
  Trigger test for all target and subtarget that have a kernel testing
  version defined.

- ci:kernel:all:first:testing
  Trigger test for all target and the first subtarget in alphabetical
  order for the target that, if they have a kernel testing version
  defined.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-06-01 19:42:21 +02:00
Petr Štetiar
95dde52329
ci: build: verify downloaded toolchain tarball
CDNs are known to ship outdated or corrupted files, if it unpacks
correctly, it necessarily doesn't mean, that we're using the desired
content. So lets fix it by checking the tarball as well.

I'm adding GPG checking explicitly, its not needed, but just double
checking, that everything is working as expected on build
infrastructure.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-31 19:32:58 +02:00
Petr Štetiar
567784127e
ci: bump buildworker container to version v6
Its being used by buildbot workers, adds g++-multilib to fix node
cross-compilation from a 64-bit build machine to 32-bit host.

References: https://github.com/openwrt/buildbot/pull/7
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-31 19:13:42 +02:00
Christian Marangi
5bafc4352f
CI: kernel: test each subtarget on push events
Test each subtarget on push events to improve testing and to refresh
ccache of each subtarget.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:05 +02:00
Christian Marangi
ebbc806d30
CI: add support for getting ccache cache from S3
Add support for getting ccache cache from S3.
ccache is archieved in a tar and downloaded from S3 Cloud Storage.

For push events, ccache is then uplodaed back to S3 to refresh and have
a ccache cache always fresh.

An additional workflow is added to upload files to an S3 Cloud Storage
from artifacts uplodaed to github. The minio tool is used to upload
files to S3.

If the ccache can't be downloaded from s3, we fallback to github cache
system.

Also limit s3 upload to the openwrt repository since external fork won't
have (obviously) the required secrtes to upload data to the S3 Cloud
Storage.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:05 +02:00
Christian Marangi
ff66a7c1c0
CI: build: limit cache save/delete only on push events
Limit ccache cache save/delete only on push events. Saving ccache
cache for pull request will result in bloat and refreshing ccache is not
possible due to security measure on enforcing read permission on
pull_request events.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:05 +02:00
Christian Marangi
2129ee1879
CI: coverity: disable ccache usage
Disable ccache usage for coverity workflow as it may cause side effect
in the produced bins.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:04 +02:00
Christian Marangi
ae7b05328c
CI: build: fix ccache cache usage
CCache cache is currently broken due to a funny bug in ccache compiler
type detection. It seems ccache compiler type detection is very fragile
and with the use of external toolchain doesn't correctly detect the
type.
The type detected is set to other instead of gcc resulting in ccache
complaining for unsupported compiler options.

To handle this problem, force the compiler type to gcc to make ccache
correctly work and speedup compilation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:04 +02:00
Christian Marangi
07b52a8a25
CI: build: add option to define custom ccache cache type
Add new input to define custom ccache cache type. This is useful to use
a different ccache cache for some special workflow that may do more test
than simple kernel compilation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:04 +02:00
Christian Marangi
b9a41c1e84
CI: build: add option to disable use of ccache
Add option to disable use of ccache. This can be useful for some
sensible test that should not use ccache as they can cause side effects
of any sort. (example Coverity Scan)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:04 +02:00
Christian Marangi
203cc0a7ef
CI: build: add job to remove previous ccache cache if already exist
Github Actions cache doesn't permit to overwrite cache if it does
already exist. As a trick to refresh and have fresh ccache pool,
delete the ccache cache if it does exist with the help of Github REST
API. An additional permission is needed to access this API. Add this
permittion to each user of the build workflow.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:03 +02:00
Christian Marangi
6321361c6b
CI: build: split cache ccache in separate restore and save jobs
Split caching ccache in separate restore and save jobs to always refresh
the ccache across different runs. Currently if a key is restored, cache
is not saved resulting in a less useful ccache that benefits from
multiple runs.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:03 +02:00
Christian Marangi
457f6b0b9c
CI: build: drop redundant generate ccache hash job
Drop redundant generare ccache hash job as that can be done by
integrated github expressions to generate an hash.
The only change is that the integrated way generate a sha256 hash
instead of an md5 sum.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-31 13:22:00 +02:00
Christian Marangi
66fd0aa6ef
CI: use toolchain container for label workflow
Use toolchain container for label workflow to skip downloading external
toolchain from openwrt servers.

Fixes: 0fe5776f4a ("CI: build: Add support to use container included external toolchain")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-28 03:45:42 +02:00
Christian Marangi
1fa84354a9
CI: don't add "" in target and subtarget for label workflow
Don't add "" in target and subtarget for label workflow from label
detection as it does cause problem in build workflow on container
target/subtarget matching.

Fixes: bf8187d5dc ("CI: use split target and subtarget in label workflow")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-27 19:53:15 +02:00
Christian Marangi
0063e71d66
CI: build: fix parse toolchain step failing for git strict rules
Commit 1cb8cdb ("ci: use new buildbot worker images with Debian 11")
introduced new Git version with strict rules for owner of the git
directory.

To handle this and not cause major change, just move the parsing before
the change of ownership of the openwrt directory permitting the correct
run of git fetch command with the same user that did the repository
checkout.

Fixes: 1cb8cdb ("ci: use new buildbot worker images with Debian 11")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-25 23:44:03 +02:00
Christian Marangi
8aa5a86010
CI: correctly output subtarget in label workflow
Commit bf8187d5dc ("CI: use split target and subtarget in label
workflow") didn't correctly output subtarget resulting in calling with
an empty subtarget. Fix this and correctly output generated subtarget.

Fixes: bf8187d5dc ("CI: use split target and subtarget in label workflow")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-25 23:09:59 +02:00
Christian Marangi
bf8187d5dc
CI: use split target and subtarget in label workflow
With eecc6e4811 ("CI: rework build workflow to have split target and
subtarget directly") target and subtarget are split in 2 different
variables. Label workflow were not aligned to this change and are
currently broken.

Fix them and correctly pass split target and subtarget.

Fixes: eecc6e4811 ("CI: rework build workflow to have split target and subtarget directly")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-25 18:24:00 +02:00
Christian Marangi
f5a5ce8822
CI: ignore master branch for push events
Due to problem with migrating from master to main as the default branch
and downstream project still requiring the master branch to be present,
we currently have for push events double CI runs, one for main and one
for master. To solve this ignore any push event to the master branch for
every workflow that react on push events.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-25 13:57:17 +02:00
Christian Marangi
0fe5776f4a
CI: build: Add support to use container included external toolchain
Add support to use container included external toolchain and skip
redownloading external sdk for each test.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-23 23:33:14 +02:00
Christian Marangi
e1370cdd49
CI: push-containers: build and push container with external toolchain
Build and push container with external toolchain embedded in the
container image.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-23 23:33:13 +02:00
Christian Marangi
23a5c715a9
CI: build: add checks to test if toolchain container can be used
Add checks to test if toolchain container can be used.
This is to handle case of new target or migration of any sort.

If the toolchain container can't be found, the tools container is used
instead.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-23 23:33:13 +02:00
Christian Marangi
803b011048
CI: build: add option to configure container to use
Add option to configure container to use for build test.
By default the tools container is used if no option is provided.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-23 23:33:10 +02:00
Christian Marangi
ce2e7c52f8
CI: build: package external toolchain after build
Package external toolchain after correct build.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-23 23:24:51 +02:00
Petr Štetiar
8fc2a0f00f ci: push-containers: trigger job on release branching
Currently all 23.05 related CI jobs are failing as the containers are
not available, so lets fix it by pushing those containers when the
version.mk changes.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-23 17:49:57 +02:00
Petr Štetiar
71ca2a3154 ci: tools: run the job on changes in include directory as well
In order to prevent regressions like with #12617.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-23 17:49:57 +02:00
Christian Marangi
eecc6e4811
CI: rework build workflow to have split target and subtarget directly
Instead of referring to a redundant job and ENV variables, rework build
workflow to accept and require split target and subtarget and use them
directly from inputs.

Rework each user and pass a JSON of tuple to matrix include with each
target/subtarget combination to test. Special notice this doesn't use
the github actions matrix combination feature but reference each
specific tuple of target and subtarget to test.

Just a cleanup no behaviour change intended.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-22 17:11:27 +02:00
Christian Marangi
6c80a578a4
CI: check-kernel-patches: use buildbot user on git diff check
Use buildbot user on git diff check instead of using git config
safe directory.

This should accomplish the same result but should be a better approach
following safe practice enforced by git.

Fixes: a7747e8670 ("ci: fix check kernel patches job")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-05-22 16:05:13 +02:00
Petr Štetiar
a7747e8670 ci: fix check kernel patches job
Currently the check fails due to the following error:

 warning: Not a git repository. Use --no-index to compare two paths outside a working tree
 usage: git diff --no-index [<options>] <path> <path>

Thats likely caused by commit 1cb8cdbf07 ("ci: use new buildbot worker
images with Debian 11") which contains a patched Git version with CVE
security fixes introduced in DLA-3239-2:

 Multiple issues were found in Git, a distributed revision control
 system. An attacker may cause other local users into executing arbitrary
 commands, leak information from the local filesystem, and bypass
 restricted shell.

 Note: Due to new security checks, access to repositories owned and
 accessed by different local users may now be rejected by Git; in case
 changing ownership is not practical, git displays a way to bypass these
 checks using the new "safe.directory" configuration entry.

So lets opt-out of this new behavior by setting `safe.directory=*` and
thus force Git to consider all Git repositories as safe regardless of
their owner, since we need to trust those sources anyway and it should
be likely more robust solution, then fiddling with filesystem
permissions.

Fixes: 1cb8cdbf07 ("ci: use new buildbot worker images with Debian 11")
References: https://www.debian.org/lts/security/2022/dla-3239-2
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-18 08:19:17 +02:00
Petr Štetiar
1cb8cdbf07 ci: use new buildbot worker images with Debian 11
Debian 10 LTS support ends on 6/2024, so it makes no sense to use it as
a base for 23.05 release, so lets switch to Debian 11 which should've
LTS support till 6/2026.

References: f2744543fa
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-05-15 20:43:47 +02:00
Petr Štetiar
9a26669510
ci: add Coverity Scan scheduled workflow
Coverity Scan is a static code analysis service focused on open source
software quality and security, so lets scan various OpenWrt components
every Friday for the start.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-04-26 17:24:50 +02:00
Christian Marangi
7643d95bb3
CI: check-kernel-patches: upload proposed refreshed patches
Upload proposed refreshed patches if the check fails.
This should help devs refresh the patches if they don't have access to a
buildroot.

Devs should ALWAYS refresh the patches before submitting and merging
commits.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-04-22 17:19:18 +02:00
Christian Marangi
a2973060ea
CI: build: disable cache of external toolchain/sdk
Our buildbot build a different external toolchain/sdk for each build.
This cause the idea of using the tar hash to cache it broken and wrong.
This makes the github cache bloated and remove space for ccache cache.

Drop cache for external toolchain/sdk as the feature is broken and cause
problems to ccache cache.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-03-22 01:07:57 +01:00
Christian Marangi
0c2b591b84
CI: kernel: skip subtarget test on non-specific target test
Reduce testing time by skipping subtarget test on non-specific target
test.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-03-22 00:36:33 +01:00
Christian Marangi
6f89a0ca20
CI: use openwrt official tools container by default
Use openwrt official tools container by default.
Fork will use openwrt tools container by default.

This can be disabled by setting the option use_openwrt_container to
false for the build.yml and check-kernel-patches.yml.

The push-containers workflow is disabled on forks. The workflow can be
reenabled by commenting the condition in push-containers.yml.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-31 16:36:57 +01:00
Petr Štetiar
ba3aa6233d ci: allow custom kernel and target jobs based on labels
Current job triggers based on matching of changed paths is quite
limited, so lets make it possible to additionally trigger manual CI jobs
by adding CI specific pull request build labels:

 * `ci:target:x86:64` label is going to trigger CI target check jobs for
    x86/64 (sub)target.

 * `ci:kernel:x86:64` label is going to trigger CI kernel check jobs for
    x86/64 (sub)target.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-01-27 11:04:07 +01:00
Christian Marangi
d5943ffed8
CI: push-containers: limit to one concurrent run
We may find in a situation where due the queue an old run finish after
the last run, resulting in the containers getting overwritten with an
old version.

Limit the push-containers workflow to one concurrent run and cancel any
run in progress.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-24 23:23:56 +01:00
Christian Marangi
d40f59825a
CI: tools: directly copy prebuilt tools in container
Directly copy prebuilt tools in container instead of creating an
archieve and extracting it later in other workflows.

Update build workflow to support this new implementation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:06 +01:00
Christian Marangi
7fe3d3670b
CI: tools: skip including dl dir in prebuilt tools tar
We can now drop the dl dir in the prebuilt tools tar as package archieve
is not a requirement anymore and won't trigger a package recompile.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:06 +01:00
Rosen Penev
183bad6641
CI: tools: add gnu-getopt to macOS CI
This used to be implicit. No longer for some reason.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-23 15:48:07 +01:00
Christian Marangi
6ca347379a
CI: kernel: enable target testing also for pull request event
Testing target changes was only set for push events. Enable this also
for pull request events to enable testing pr making specific target
changes.

Fixes: 57a02cbbff ("CI: kernel: test each target with additional changes than target/linux")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-20 15:14:53 +01:00