Skip to content
Snippets Groups Projects

[freedom-maker]: delete support for non-free hardware and emulators v0.26

Merged bill-auger requested to merge bill-auger/package-helpers:nabia into nabia

the 'freedom-maker' package in the trisquel repos is not FSDG-fit - it's job is to build debian freedombox systems directly from the debian repos, for a variety of SBCs, some of which require non-free packages in the 'contrib' and 'non-free' repos - it also requires virtualbox for some bulid tartgets

so far, ive done only the FSDG and trisquel-ize patching - this MR may yet need changes

Edited by Luis Guzmán

Merge request reports

Approval is optional

Merged by Luis GuzmánLuis Guzmán 2 years ago (Aug 23, 2022 2:18pm UTC)

Merge details

  • Changes merged into nabia with 9753b278 (commits were squashed).
  • Did not delete the source branch.

Pipeline #645 passed

Pipeline passed for 9753b278 on nabia

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • :x: Jenkins Build FAILURE

    Results available at: Jenkins [gitlab-merge-request #902]

  • bill-auger added 2 commits

    added 2 commits

    Compare with previous version

  • :x: Jenkins Build FAILURE

    Results available at: Jenkins [gitlab-merge-request #903]

  • bill-auger added 1 commit

    added 1 commit

    • 132b3d97 - [freedom-maker]: initial patches for v0.26

    Compare with previous version

  • :x: Jenkins Build FAILURE

    Results available at: Jenkins [gitlab-merge-request #905]

  • :white_check_mark: Source Build SUCCESS

    Results available at: https://jenkins.trisquel.org/job/gitlab-merge-request/906/

    Note, this comment only means that the source build was successful, to know if the binary build was successful please check https://jenkins.trisquel.org/job/gitlab-merge-request/906/promotion/

  • bill-auger added 1 commit
  • :white_check_mark: Source Build SUCCESS

    Results available at: https://jenkins.trisquel.org/job/gitlab-merge-request/907/

    Note, this comment only means that the source build was successful, to know if the binary build was successful please check https://jenkins.trisquel.org/job/gitlab-merge-request/907/promotion/

  • Author Contributor

    just an update for documentation purposes - i got a bit stuck with the build (building the freedombox image, not the trisquel package) - i dont foresee any problems with the package - it is only python scripts - but the program itself does not work; due to a problem related to the 'flash-kernel' package

    the build succeeds almost completely but fails when installing the bootloader - the failing command is: apt-get install -y flash-kernel

    Couldn't find DTB sun7i-a20-olinuxino-lime2.dtb on the following paths: /etc/flash-kernel/dtbs /usr/lib/linux-image- /lib/firmware//device-tree/
    Installing into /boot/dtbs//./sun7i-a20-olinuxino-lime2.dtb

    those error messages emanate from the flash-kernel script, which runs as a kernel install hook: /etc/kernel/postinst.d/zz-generate-dtb-link

    when that hook runs, it expects the kernel flavor and version to be passed in, presumably the package manager does that - or else it runs the kernel-version command to get it - im not certain which is failing; but the $kver variable, which determines the input and output directory names is apparently empty, as evident in the error messages

    the debian kernel puts the .dtb files under:

    /usr/lib/linux-image-${flavor}-${version}

    but the flash-kernel script fails to locate them (understandably) under:

    /usr/lib/linux-image-

    the script should probably exit at that point; but it continues to try installing the non-existing .dtb files to:

    /boot/dtbs//./sun7i-a20-olinuxino-lime2.dtb

    which is also wrong - it should be:

    /boot/dtbs/${flavor}-${version}/./sun7i-a20-olinuxino-lime2.dtb

    so even if i hackishly put the .dtb files under the valid directory /etc/flash-kernel/dtbs, it would still fail to install them properly

    it could be because that happens in a cross-arch chroot, and is confused about which kernel is actually running - but that is how the program is designed; so i doubt that is the problem - i am more suspecting the apt hook; though the 'flash-kernel' package install works as expected on a bare-metal ARM debian system

    the next thing i would try is to verify if the 'flash-kernel' package install works as expected on a bare-metal ARM trisquel system - regardless if that works or not, i would need to build a custom flash-kernel package for trisquel with some debugging traces, to isolate exactly where it goes wrong (failing to set $kver properly)

    Edited by bill-auger
  • Luis Guzmán marked this merge request as draft

    marked this merge request as draft

  • Luis Guzmán marked this merge request as ready

    marked this merge request as ready

  • This pending bill-auger/package-helpers!1 (closed) fixes building issues with the testing phase (and also rebase), please merge and rebase.

    Regards.

  • bill-auger added 32 commits

    added 32 commits

    Compare with previous version

  • :x: Jenkins Build FAILURE

    Results available at: Jenkins [gitlab-merge-request #1,100]

  • Author Contributor

    from 2df932d5 - freedom-maker: Set valid kernel and replace suggestion:

    sed -i "s|virtualbox|debootstrap|" debian/control

    this change, i absorbed into the existing patch which removed support for virtualbox - there are still other remnants of virtualbox support remaining (tests and the man page, IIRC), which could be added to patch 0006 eventually

    into 3bad2aca - freedom-maker: remove non-free suggestion from packaging metadata


    from 2df932d5 - freedom-maker: Set valid kernel and replace suggestion:

    sed -i "s|'linux-image-' + self.builder.kernel_flavor|'linux-image-generic'|" freedommaker/internal.py

    this change, i already had a partial patch for - the kernel flavor is a proper data member per-class of CPU; so this is the proper way to handle the change

    into 2932c7d2 - freedom-maker: specify generic armhf kernel

    0008-specify-generic-armhf-kernel.patch:

    diff --git a/freedommaker/builders/a20.py b/freedommaker/builders/a20.py
    index f543ae5..7f13c6d 100644
    --- a/freedommaker/builders/a20.py
    +++ b/freedommaker/builders/a20.py
    @@ -25,7 +25,7 @@ from .arm import ARMImageBuilder
     class A20ImageBuilder(ARMImageBuilder):
         """Base image builder for all Allwinner A20 board based targets."""
         architecture = 'armhf'
    -    kernel_flavor = 'armmp-lpae'
    +    kernel_flavor = 'generic'
         boot_offset = '1mib'
         u_boot_path = None
     
    diff --git a/freedommaker/builders/beaglebone.py b/freedommaker/builders/beaglebone.py
    index 99a2d40..aca4a78 100644
    --- a/freedommaker/builders/beaglebone.py
    +++ b/freedommaker/builders/beaglebone.py
    @@ -26,7 +26,7 @@ class BeagleBoneImageBuilder(ARMImageBuilder):
         """Image builder for BeagleBone target."""
         architecture = 'armhf'
         machine = 'beaglebone'
    -    kernel_flavor = 'armmp'
    +    kernel_flavor = 'generic'
         boot_offset = '2mib'
         flash_kernel_name = 'TI AM335x BeagleBone Black'
         flash_kernel_options = 'console=ttyO0'

    the question this raises though (i could probably deduce this myself) is which flavor to specify for the other targets - the sed solution restricted all targets to the 'generic' kernel - the current version of patch 0008 only addresses the supported ARM boards - the current values will be so:

    $ grep -r kernel_flavor freedommaker/builders/
    freedommaker/builders/a64.py:         kernel_flavor = 'arm64'
    freedommaker/builders/armhf.py:       kernel_flavor = 'armmp-lpae'
    freedommaker/builders/a20.py:         kernel_flavor = 'generic'
    freedommaker/builders/qemu_i386.py:   kernel_flavor = '686'
    freedommaker/builders/i386.py:        kernel_flavor = '686'
    freedommaker/builders/amd64.py:       kernel_flavor = 'amd64'
    freedommaker/builders/arm64.py:       kernel_flavor = 'arm64'
    freedommaker/builders/beaglebone.py:  kernel_flavor = 'generic'
    freedommaker/builders/qemu_amd64.py:  kernel_flavor = 'amd64'

    maybe some of those would need to change also (added to patch 0008), depending on which kernels are available - are all trisquel kernels the 'generic' flavor?

    trisquel no longer supports i386, yes? - so maybe another patch is needed to remove the i386 classes, similar to the one that removes the virtualbox classes

    (i just noticed that i neglected to treat the generic 'armhf' target, the same as 'a20' and 'beaglebone') - still, this program does not produce a useful ARM image - until the problem with the 'flash-kernel' package is resolved, it supports no ARM machines actually


    from e5e9da22 - freedom-maker: tweak patch application:

    + for patch in $(find $DATA/ -name *.patch)

    the third change addresses this bug:

    $ find  -name *.patch
    find: paths must precede expression: `0002-generalize-distro-fork-name_could-be-upstreamed.patch'
    find: possible unquoted pattern after predicate `-name'?

    in 6dcd9cd4 - freedom-maker: bugfix patch find command

    Edited by bill-auger
  • Luis Guzmán
  • depending on which kernels are available - are all trisquel kernels the 'generic' flavor

    AFAIK, all architectures on ubuntu/trisquel have a "generic" kernel, trisquel maintain generic and generic-hwe, which contain the most updates and changes upstream.

    trisquel no longer supports i386, yes?

    This is true for nabia, there are aims to bring it back for aramo, hopefully the current dev team can take over that challenge.

    still, this program does not produce a useful ARM image

    This is expected, as there is no officially supported devices, and still very early trisquel testing on ARM devices. We might temporary remove arm support 'for ARM and update once this arch get further development.

    Cheers!

  • bill-auger added 3 commits

    added 3 commits

    • 9fb80a97 - freedom-maker: specify generic kernels
    • 1acbac6f - freedom-maker: remove non-free suggestion from packaging metadata
    • 304cad5b - freedom-maker: bugfix patch find command

    Compare with previous version

  • :x: Jenkins Build FAILURE

    Results available at: Jenkins [gitlab-merge-request #1,101]

  • Author Contributor

    ok i tweaked the kernel flavors patch - it is now 0008-specify-generic-kernels.patch

    i will add this patch, just for documentation of the current blocker

    diff --git a/freedommaker/library.py b/freedommaker/library.py
    index d724053..ef92da8 100644
    --- a/freedommaker/library.py
    +++ b/freedommaker/library.py
    @@ -526,6 +526,44 @@ def setup_flash_kernel(state, machine_name, kernel_options,
             run_in_chroot(
                 state, ['debconf-set-selections'], feed_stdin=stdin.encode())
     
    +    # FIXME: BEGIN - prepare missing device-tree blobs
    +    #        the needed *.dtb files would normally be under /usr/lib/linux-image-<KERNEL_VER>/;
    +    #        however there are currently two problems:
    +    #        * the trisquel kernel build does not produce or package them
    +    #          -> adding them manually for now
    +    #        * the `flash-kernel` script will not find them at the standard location,
    +    #          because it's $kver variable is empty when used
    +    #          -> "Couldn't find DTB sun7i-a20-olinuxino-lime2.dtb on the following paths:
    +    #              /etc/flash-kernel/dtbs  /usr/lib/linux-image-  /lib/firmware//device-tree/"
    +    #          NOTE: the empty $kver variable is inserted here  ^     and here ^
    +    #
    +    #          however, even with this kludge, the `flash-kernel` script
    +    #          will still fail to install them, for the same reason
    +    #          -> /boot/dtbs//./sun7i-a20-olinuxino-lime2.dtb
    +    #          NOTE:        ^ <-- the empty $kver variable is inserted here
    +    #          it should be:
    +    #          -> /boot/dtbs/${flavor}-${version}/./sun7i-a20-olinuxino-lime2.dtb
    +    #
    +    # compile device-tree blobs
    +    kernel_id = run_in_chroot(state, ['kernel-version', 'list']).decode()
    +    kernel_ver = kernel_id.split('-')[0]
    +    kernel_flavor = kernel_id.removeprefix(kernel_ver)
    +    #kernel_flavor = '-generic'
    +    run(['apt' 'source'    'linux-image' % kernel_flavor])
    +    run(['apt' 'build-dep' 'linux-image' % kernel_flavor])
    +    run(['apt' 'install' 'gcc-arm-linux-gnueabihf'])
    +    run(['cd' 'linux-%s/' % kernel_ver]) or raise "kernel version mismatch"
    +    run(['make' 'mrproper'])
    +    run(['make' 'ARCH=arm' 'sunxi_defconfig'])
    +    run(['make' 'ARCH=arm' 'CROSS_COMPILE=arm-linux-gnueabihf-' 'dtbs'])
    +    #
    +    # install device-tree blobs
    +    dtbs_src_dir = 'arch/arm/boot/dts/'
    +    #dtbs_dest_dir = path_in_mount(state, 'usr/lib/linux-image-%s/' % kernel_id)
    +    dtbs_dest_dir = path_in_mount(state, 'etc/flash-kernel/dtbs/')
    +    shutil.copytree(dtbs_src_dir, dtbs_dest_dir)
    +    # FIXME: END
    +
         run_in_chroot(state, ['apt-get', 'install', '-y', 'flash-kernel'])
     
         # flash-kernel creates links in /boot and does not work with the filesystem
    
    Edited by bill-auger
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading