Commit Graph

21 Commits

Author SHA1 Message Date
Paul Spooren
49e8f53298 scripts: fix installed-size calculation
Previously the script would calculate the size of the compressed archive
which isn't the size installed in the overlayfs.

This commit uses zcat in combination with wc to calculate the
umcompressed size.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2024-01-08 14:08:06 +01:00
Paul Spooren
7a73221332 build: use numeric-owner in ipkg-build
To create packages the `ipkg-build` script is used which double packs
`control.tar.gz` and `data.tar.gz` to a single package. By default it's
using a verbose username instead of a numeric value for files.

Official OpenWrt images (artifacts) are created within docker containers
which do not seem to contain those verbose usernames and instead
defaults to numeric values.

This becomes a problem when rebuilding public artifacts because other
build environments may offer verbose usernames and there the created
packages is different from the official ones.

With this commit `ipkg-build` always uses numeric values for user/group
and thereby making it easier to reproduce official artifacts.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-19 22:59:50 +02:00
Daniel Golle
1d77dca3b3
scripts: make sure sort-order is independent from locale
Set LC_ALL=C environment variable when calling 'sort' as the sort
order otherwise depends on the locale set.

Fixes: 56ce110b73 ("scripts: make sure conffiles are sorted")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-04-06 17:33:31 +01:00
Paul Spooren
56ce110b73 scripts: make sure conffiles are sorted
It may happen that conffiles are in different order on different builds.
Make sure they have the same order by sorting them.

FIX: #9612

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-01 13:08:58 +01:00
Paul Spooren
bb95be9265 scripts,ipkg-build: use realpath for pkg_dir
This allows manual execution of the ipkg-build script even with
releative path.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:32:30 -10:00
Paul Spooren
72f481b157 scripts,ipkg-build: apply shellcheck
This commit cleans the `ipkg-build` script via changes suggested by
shellcheck. These are mostly word splitting issues.

Remove the definition of GZIP, this adds three "lookups" of the `gzip`
binary but the rest of the build system doesn't seem to use such
improvements neither.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:31:43 -10:00
Jo-Philipp Wich
4038c031cb scripts: ipkg-build: simplify uid/gid resolving
Use the prepared .packageusergroup file to lookup user and group names
when processing the passed file mode.

Also replace the various subshell/cut invocations with a sequence of
standard variable interpolations which fixes paths with embedded colons
as a side-effect.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-14 17:11:04 +02:00
Paul Spooren
51ec51871f build: add user/group ID resolve function
With the introduction of `./tmp/userids` the `ipkg-build` script can now
resolve values of "PKG_FILE_MODES", allowing users to set names rather
than numeric values.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-09-14 10:54:52 +01:00
Paul Spooren
fd29f2bcd9 build: store granular timestamps in packages
With the new `SOURCE` argument of `get_source_date_epoch` it is possible
to set package timestamps based on actual package changes rather thane
$TOPDIR changes.

This commit adds a new variable PKG_SOURCE_DATE_EPOCH which is used by
the `ipkg` build script. As a fallback the existing SOURCE_DATE_EPOCH is
used or as last resort the current time.

The redundant checks for `.git/` and `.svn/` are removed.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:18:06 +01:00
Paul Spooren
353ce2e521 build: ipkg-build use fakeroot with PKG_FILE_MODES
The `ipkg-build` script converts a folder into a `opkg` installable
package. Until now it would use root:root for all packages and try to
preserve file modes.

This has the two drawbacks of packages want to add non-root files or add
SUID files, like the `sudo` package does.

To give more flexibility regarding file modes and avoid init script
hacks, a new variable called `PKG_FILE_MODES`. The variable contains a
list of files modes in the format `path:owner:group:mode`.

An example for the `sudo` package below:

```
PKG_FILE_MODES:=\
        /usr/bin/sudo:root:root:4755 \
        /etc/sudoers:root:root:0440
```

The `ipkg-build` now runs within a fakeroot environment to set any mode
and directly store it in the resulting `ipk` package archive.

Both options `-o` and `-g` are no longer required due to the introduction
of the more flexible `-m` options, which takes the `PKG_FILE_MODES` as
input.

Lastly the option `-c` is removed as it's unused within the script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-08-31 11:13:12 +01:00
Paul Spooren
1fdf6b745c treewide: replace which with command -v
Fix shellcheck SC2230
> which is non-standard. Use builtin 'command -v' instead.

Using `command -v` is POSIX compliant while `which` is not.  Also to
mention, `command -v` is a shell builtin whereas `which` is a separate
busybox applet.

Once applied to everything concerning OpenWrt we can disable the busybox
feature `which` and save 3.8kB.

Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
[also replace cases in zram-swap]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2020-08-12 11:08:05 +02:00
Jo-Philipp Wich
d157a76c67 script: ipkg-build: honour $SOURCE_DATE_EPOCH
When the SOURCE_DATE_EPOCH environment variable is set, use it to
override the timestamps of .ipk archive contents.

This ensures that .ipk archives built in environments without SCM
metadata (mainly the SDK) are reproducible between different runs.

Ref: https://github.com/openwrt/packages/issues/6954
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-11-08 11:52:33 +01:00
Yousong Zhou
9e58c20ec9 scripts/ipkg-build: quash error messages when conffiles.resolved is empty
When Package/xx/conffiles only contains directories that are empty at
package time, conffiles.resolved will be missing and the following
error messages will appear in the build log.

	/home/yousong/git-repo/openwrt/openwrt/scripts/ipkg-build -c -o 0 -g 0 /home/yousong/git-repo/openwrt/openwrt/build_dir/target-mips_24kc_musl/linux-malta_be/openvswitch-2.10.0/ipkg-mips_24kc/openvswitch-common /home/yousong/git-repo/openwrt/openwrt/bin/packages/mips_24kc/packages
	mv: cannot stat 'CONTROL/conffiles.resolved': No such file or directory
	chmod: cannot access 'CONTROL/conffiles': No such file or directory

It will not break the ipkg-build process.  The change is merely cosmetic
to not cause confusion when reading logs

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2018-09-09 01:21:01 +00:00
Jo-Philipp Wich
3a3424981c scripts: ipkg-build: do not require git or svn
Move the "which svn" and "which git" calls next to the timestamp commands
using those tools to not prematurely fail on systems where svn or git are
not present.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-15 13:33:32 +02:00
Syrone Wong
e8bc0834e7 scripts/ipkg-build: fix deprecated GZIP environment variable warning
According to gzip 1.7 release note:

The GZIP environment variable is now obsolescent; gzip now warns if
it is used, and rejects attempts to use dangerous options or operands.
You can use an alias or script instead.

Fix this warning by using pipe instead

Signed-off-by: Syrone Wong <wong.syrone@gmail.com>
2016-05-13 17:03:53 +02:00
Felix Fietkau
1a2fd4f869 ipkg-build: add ogargs to outer tar/ipk
prevent name:uid leaks of build system

Signed-off-by: Dirk Neukirchen <dirkneukirchen@web.de>

SVN-Revision: 46863
2015-09-11 16:34:22 +00:00
Felix Fietkau
4baec2468a include/image.mk: make tar reproducible
several packages reported different file order between builds
make binutils, kmod-sched reproducible

Signed-off-by: Dirk Neukirchen <dirkneukirchen@web.de>

SVN-Revision: 46859
2015-09-11 16:33:26 +00:00
Felix Fietkau
7f359e52ce scripts/ipkg-build: fix a build regression introduced in r46360 (fixes #20082)
go to the $TOPDIR directory before trying to extract a git/svn timestamp

Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 46383
2015-07-15 08:17:49 +00:00
John Crispin
3846bb9ac9 ipkg-build: use deterministic timestamps in ipk tarballs
As a follow-up to r46026, this commit sets the modification
times of files inside the tarballs to deterministic values.
It uses the date of the last git or svn commit and falls
back to the old behavior if those are not available.

Signed-off-by: Reiner Herrmann <reiner@reiner-h.de>

SVN-Revision: 46360
2015-07-14 09:57:19 +00:00
Felix Fietkau
949d7e30e4 ipkg-build: suppress timestamps in gzip headers
When creating control.tar.gz, data.tar.gz and $package.ipk
tar compresses them with default gzip options.
This includes build timestamps in their gzip headers.
By setting the GZIP env variable to -n, gzip no longer embeds
them, which is a step towards reproducibility.

See also [1] for more detailed motivations.

[1]: https://wiki.debian.org/ReproducibleBuilds/About

Signed-off-by: Reiner Herrmann <reiner@reiner-h.de>

SVN-Revision: 46026
2015-06-18 06:41:45 +00:00
Felix Fietkau
55c5d10ca9 tools: replace ipkg-utils with a reduced ipkg-build variant in scripts/
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45129
2015-03-29 07:35:26 +00:00